[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474496157106-28370-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:54:56 dist-test-slave-2j7r krb5kdc[28383](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:54:56 dist-test-slave-2j7r krb5kdc[28383](info): set up 2 sockets
Aug 11 18:54:56 dist-test-slave-2j7r krb5kdc[28383](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260811 18:54:57.421553 28370 mini_kdc.cc:121] Time spent starting KDC: real 1.188s user 0.003s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474496157106-28370-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474496157106-28370-0/krb5kdc/test-user.keytab.
Aug 11 18:54:57 dist-test-slave-2j7r krb5kdc[28383](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474497, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:54:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:54:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.180.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.180.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.180.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab.
Entry for principal kudu/127.27.180.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab.
I20260811 18:54:57.549144 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474496157106-28370-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474496157106-28370-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474496157106-28370-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.190:43381
--webserver_interface=127.27.180.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab
--principal=kudu/127.27.180.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:44859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.180.190:43381
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:54:57.671025 28408 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:54:57.671267 28408 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:54:57.671306 28408 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:54:57.675199 28408 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:54:57.675248 28408 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:54:57.675266 28408 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:54:57.675282 28408 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:54:57.675297 28408 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:54:57.680815 28408 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:44859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474496157106-28370-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474496157106-28370-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.180.190:43381
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab
--principal=kudu/127.27.180.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.190:43381
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.180.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28408
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474496157106-28370-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:54:57.682376 28408 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:54:57.683749 28408 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:54:57.698127 28415 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:54:57.698146 28417 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:54:57.699156 28408 server_base.cc:1034] running on GCE node
W20260811 18:54:57.700773 28419 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:54:57.701594 28408 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:54:57.703510 28408 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:54:57.704699 28408 hybrid_clock.cc:648] HybridClock initialized: now 1786474497704664 us; error 83 us; skew 500 ppm
Aug 11 18:54:57 dist-test-slave-2j7r krb5kdc[28383](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474497, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:54:57.712158 28408 init.cc:377] Logged in from keytab as kudu/127.27.180.190@KRBTEST.COM (short username kudu)
I20260811 18:54:57.713482 28408 webserver.cc:460] Webserver started at http://127.27.180.190:35783/ using document root <none> and password file <none>
I20260811 18:54:57.714453 28408 fs_manager.cc:359] Metadata directory not provided
I20260811 18:54:57.714529 28408 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:54:57.714946 28408 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:54:57.717677 28408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474496157106-28370-0/minicluster-data/master-0/data/instance:
uuid: "f681fc9435014685b9b5f87f31769a5a"
format_stamp: "Formatted at 2026-08-11 18:54:57 on dist-test-slave-2j7r"
server_key: "b636caf63d1e30605b4d348b0a369ef8"
server_key_iv: "28f3caa9e62a9b26e6a9d0d87ee89c65"
server_key_version: "encryptionkey@0"
I20260811 18:54:57.718238 28408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474496157106-28370-0/minicluster-data/master-0/wal/instance:
uuid: "f681fc9435014685b9b5f87f31769a5a"
format_stamp: "Formatted at 2026-08-11 18:54:57 on dist-test-slave-2j7r"
server_key: "b636caf63d1e30605b4d348b0a369ef8"
server_key_iv: "28f3caa9e62a9b26e6a9d0d87ee89c65"
server_key_version: "encryptionkey@0"
I20260811 18:54:57.723453 28408 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260811 18:54:57.727375 28432 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:54:57.729206 28408 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 18:54:57.729419 28408 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474496157106-28370-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474496157106-28370-0/minicluster-data/master-0/wal
uuid: "f681fc9435014685b9b5f87f31769a5a"
format_stamp: "Formatted at 2026-08-11 18:54:57 on dist-test-slave-2j7r"
server_key: "b636caf63d1e30605b4d348b0a369ef8"
server_key_iv: "28f3caa9e62a9b26e6a9d0d87ee89c65"
server_key_version: "encryptionkey@0"
I20260811 18:54:57.729539 28408 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474496157106-28370-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474496157106-28370-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474496157106-28370-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:54:57.743394 28408 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:54:57.746778 28408 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:54:57.746996 28408 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:54:57.756511 28408 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.190:43381
I20260811 18:54:57.756559 28526 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.190:43381 every 8 connection(s)
I20260811 18:54:57.757853 28408 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
I20260811 18:54:57.762993 28527 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:54:57.766649 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 28408
I20260811 18:54:57.766814 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474496157106-28370-0/minicluster-data/master-0/wal/instance
I20260811 18:54:57.768631 28370 external_mini_cluster.cc:1428] Setting key 9c1ce0dc17341a4a71671ea1201cb4d2
I20260811 18:54:57.773633 28527 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f681fc9435014685b9b5f87f31769a5a: Bootstrap starting.
I20260811 18:54:57.777503 28527 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f681fc9435014685b9b5f87f31769a5a: Neither blocks nor log segments found. Creating new log.
I20260811 18:54:57.778702 28527 log.cc:824] T 00000000000000000000000000000000 P f681fc9435014685b9b5f87f31769a5a: Log is configured to *not* fsync() on all Append() calls
I20260811 18:54:57.781553 28527 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f681fc9435014685b9b5f87f31769a5a: No bootstrap required, opened a new log
I20260811 18:54:57.785851 28527 raft_consensus.cc:348] T 00000000000000000000000000000000 P f681fc9435014685b9b5f87f31769a5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f681fc9435014685b9b5f87f31769a5a" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 43381 } }
I20260811 18:54:57.786183 28527 raft_consensus.cc:374] T 00000000000000000000000000000000 P f681fc9435014685b9b5f87f31769a5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:54:57.786283 28527 raft_consensus.cc:729] T 00000000000000000000000000000000 P f681fc9435014685b9b5f87f31769a5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f681fc9435014685b9b5f87f31769a5a, State: Initialized, Role: FOLLOWER
I20260811 18:54:57.787001 28527 consensus_queue.cc:260] T 00000000000000000000000000000000 P f681fc9435014685b9b5f87f31769a5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f681fc9435014685b9b5f87f31769a5a" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 43381 } }
I20260811 18:54:57.787201 28527 raft_consensus.cc:388] T 00000000000000000000000000000000 P f681fc9435014685b9b5f87f31769a5a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:54:57.787328 28527 raft_consensus.cc:482] T 00000000000000000000000000000000 P f681fc9435014685b9b5f87f31769a5a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:54:57.787433 28527 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f681fc9435014685b9b5f87f31769a5a [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:54:57.788471 28527 raft_consensus.cc:504] T 00000000000000000000000000000000 P f681fc9435014685b9b5f87f31769a5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f681fc9435014685b9b5f87f31769a5a" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 43381 } }
I20260811 18:54:57.789168 28527 leader_election.cc:302] T 00000000000000000000000000000000 P f681fc9435014685b9b5f87f31769a5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f681fc9435014685b9b5f87f31769a5a; no voters:
I20260811 18:54:57.789534 28527 leader_election.cc:288] T 00000000000000000000000000000000 P f681fc9435014685b9b5f87f31769a5a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:54:57.789685 28535 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f681fc9435014685b9b5f87f31769a5a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:54:57.789932 28535 raft_consensus.cc:686] T 00000000000000000000000000000000 P f681fc9435014685b9b5f87f31769a5a [term 1 LEADER]: Becoming Leader. State: Replica: f681fc9435014685b9b5f87f31769a5a, State: Running, Role: LEADER
I20260811 18:54:57.790426 28535 consensus_queue.cc:237] T 00000000000000000000000000000000 P f681fc9435014685b9b5f87f31769a5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f681fc9435014685b9b5f87f31769a5a" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 43381 } }
I20260811 18:54:57.790975 28527 sys_catalog.cc:564] T 00000000000000000000000000000000 P f681fc9435014685b9b5f87f31769a5a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:54:57.792344 28535 sys_catalog.cc:455] T 00000000000000000000000000000000 P f681fc9435014685b9b5f87f31769a5a [sys.catalog]: SysCatalogTable state changed. Reason: New leader f681fc9435014685b9b5f87f31769a5a. Latest consensus state: current_term: 1 leader_uuid: "f681fc9435014685b9b5f87f31769a5a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f681fc9435014685b9b5f87f31769a5a" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 43381 } } }
Aug 11 18:54:57 dist-test-slave-2j7r krb5kdc[28383](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474497, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM
I20260811 18:54:57.792526 28535 sys_catalog.cc:458] T 00000000000000000000000000000000 P f681fc9435014685b9b5f87f31769a5a [sys.catalog]: This master's current role is: LEADER
I20260811 18:54:57.793068 28549 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:54:57.793408 28537 sys_catalog.cc:455] T 00000000000000000000000000000000 P f681fc9435014685b9b5f87f31769a5a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f681fc9435014685b9b5f87f31769a5a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f681fc9435014685b9b5f87f31769a5a" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 43381 } } }
I20260811 18:54:57.793757 28537 sys_catalog.cc:458] T 00000000000000000000000000000000 P f681fc9435014685b9b5f87f31769a5a [sys.catalog]: This master's current role is: LEADER
I20260811 18:54:57.798532 28549 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:54:57.800092 28531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:57.774239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44930 (local address 127.27.180.190:43381)
0811 18:54:57.774726 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:57.774754 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:57.777771 (+ 3017us) server_negotiation.cc:407] Connection header received
0811 18:54:57.781343 (+ 3572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:57.781382 (+ 39us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:57.781920 (+ 538us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:57.782280 (+ 360us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:57.784429 (+ 2149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:57.786613 (+ 2184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:57.787920 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:57.788589 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:57.788753 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:57.793743 (+ 4990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:54:57.793782 (+ 39us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:54:57.793800 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:54:57.793832 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:54:57.795943 (+ 2111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:57.796588 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:57.796594 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:57.796599 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:57.796749 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:57.797189 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:57.797192 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:57.797194 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:57.797812 (+ 618us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:54:57.798012 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:57.798706 (+ 694us) server_negotiation.cc:299] Negotiation successful
0811 18:54:57.799013 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":122,"threads_started":1}
I20260811 18:54:57.806748 28549 catalog_manager.cc:1347] Generated new cluster ID: 2f5f93e6097c45f1a3114b01ddacbca0
I20260811 18:54:57.806838 28549 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:54:57.822186 28549 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:54:57.823505 28549 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:54:57.829702 28549 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f681fc9435014685b9b5f87f31769a5a: Generated new TSK 0
I20260811 18:54:57.830494 28549 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.180.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.180.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.180.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.129.keytab.
Entry for principal kudu/127.27.180.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.129.keytab.
I20260811 18:54:57.862322 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474496157106-28370-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474496157106-28370-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474496157106-28370-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474496157106-28370-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.180.129:0
--local_ip_for_outbound_sockets=127.27.180.129
--webserver_interface=127.27.180.129
--webserver_port=0
--tserver_master_addrs=127.27.180.190:43381
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.129.keytab
--principal=kudu/127.27.180.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:44859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:54:58.022336 28577 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:54:58.022588 28577 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:54:58.022631 28577 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:54:58.026682 28577 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:54:58.026734 28577 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:54:58.026794 28577 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.180.129
I20260811 18:54:58.032199 28577 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:44859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474496157106-28370-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474496157106-28370-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.129.keytab
--principal=kudu/127.27.180.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.180.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474496157106-28370-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.180.129
--webserver_port=0
--tserver_master_addrs=127.27.180.190:43381
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28577
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.180.129
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474496157106-28370-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:54:58.033293 28577 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:54:58.034338 28577 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:54:58.045264 28587 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:54:58.045320 28583 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:54:58.045310 28585 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:54:58.045569 28577 server_base.cc:1034] running on GCE node
I20260811 18:54:58.045962 28577 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:54:58.046553 28577 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:54:58.047717 28577 hybrid_clock.cc:648] HybridClock initialized: now 1786474498047697 us; error 30 us; skew 500 ppm
Aug 11 18:54:58 dist-test-slave-2j7r krb5kdc[28383](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474498, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:54:58.051076 28577 init.cc:377] Logged in from keytab as kudu/127.27.180.129@KRBTEST.COM (short username kudu)
I20260811 18:54:58.051905 28577 webserver.cc:460] Webserver started at http://127.27.180.129:45999/ using document root <none> and password file <none>
I20260811 18:54:58.052435 28577 fs_manager.cc:359] Metadata directory not provided
I20260811 18:54:58.052501 28577 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:54:58.052685 28577 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:54:58.054699 28577 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474496157106-28370-0/minicluster-data/ts-0/data/instance:
uuid: "940df53837ca4218bac0550cbcdfc982"
format_stamp: "Formatted at 2026-08-11 18:54:58 on dist-test-slave-2j7r"
server_key: "fff0ae9bfd9d0dec67a5a154d037cba7"
server_key_iv: "a3c2589790a88f228ff379152d98361e"
server_key_version: "encryptionkey@0"
I20260811 18:54:58.055274 28577 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474496157106-28370-0/minicluster-data/ts-0/wal/instance:
uuid: "940df53837ca4218bac0550cbcdfc982"
format_stamp: "Formatted at 2026-08-11 18:54:58 on dist-test-slave-2j7r"
server_key: "fff0ae9bfd9d0dec67a5a154d037cba7"
server_key_iv: "a3c2589790a88f228ff379152d98361e"
server_key_version: "encryptionkey@0"
I20260811 18:54:58.059744 28577 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260811 18:54:58.062326 28599 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:54:58.063573 28577 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:54:58.063681 28577 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474496157106-28370-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474496157106-28370-0/minicluster-data/ts-0/wal
uuid: "940df53837ca4218bac0550cbcdfc982"
format_stamp: "Formatted at 2026-08-11 18:54:58 on dist-test-slave-2j7r"
server_key: "fff0ae9bfd9d0dec67a5a154d037cba7"
server_key_iv: "a3c2589790a88f228ff379152d98361e"
server_key_version: "encryptionkey@0"
I20260811 18:54:58.063786 28577 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474496157106-28370-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474496157106-28370-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474496157106-28370-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:54:58.099483 28577 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:54:58.103257 28577 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:54:58.103513 28577 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:54:58.104517 28577 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:54:58.105871 28577 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:54:58.105937 28577 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:54:58.105990 28577 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:54:58.106009 28577 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:54:58.126181 28577 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.129:42007
I20260811 18:54:58.126346 28838 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.129:42007 every 8 connection(s)
I20260811 18:54:58.127139 28577 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474496157106-28370-0/minicluster-data/ts-0/data/info.pb
I20260811 18:54:58.129604 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 28577
I20260811 18:54:58.129709 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474496157106-28370-0/minicluster-data/ts-0/wal/instance
I20260811 18:54:58.129949 28370 external_mini_cluster.cc:1428] Setting key d5da84b1d7b727c64d8f8b7efa1de18d
Aug 11 18:54:58 dist-test-slave-2j7r krb5kdc[28383](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474498, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.129@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.27.180.130@KRBTEST.COM; defaulting to no policy
I20260811 18:54:58.143767 28531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:58.129559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.129:36463 (local address 127.27.180.190:43381)
0811 18:54:58.129715 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:58.129720 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:58.131349 (+ 1629us) server_negotiation.cc:407] Connection header received
0811 18:54:58.132528 (+ 1179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:58.132531 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:58.132583 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:58.132664 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:58.134536 (+ 1872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:58.135081 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:58.136915 (+ 1834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:58.137238 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:58.137319 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:58.140323 (+ 3004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:54:58.140328 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:54:58.140331 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:54:58.140359 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:54:58.141906 (+ 1547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:58.142424 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:58.142428 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:58.142431 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:58.142518 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:58.142784 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:58.142786 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:58.142787 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:58.142945 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:54:58.143015 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:58.143521 (+ 506us) server_negotiation.cc:299] Negotiation successful
0811 18:54:58.143636 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 18:54:58.144670 28843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:58.129952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:43381 (local address 127.27.180.129:36463)
0811 18:54:58.131187 (+ 1235us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:58.131245 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 18:54:58.132324 (+ 1079us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:54:58.132814 (+ 490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:54:58.132820 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:54:58.133549 (+ 729us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:54:58.134339 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:58.134361 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:58.135214 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:58.135218 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:58.136794 (+ 1576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:58.136807 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:58.137394 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:58.137399 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:58.137682 (+ 283us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:58.138304 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:54:58.138326 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:54:58.140150 (+ 1824us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:54:58.142027 (+ 1877us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:54:58.142032 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:54:58.142042 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:54:58.142324 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:54:58.142590 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:54:58.142592 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:54:58.142593 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:54:58.142717 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:54:58.143015 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:54:58.143020 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:54:58.143297 (+ 277us) client_negotiation.cc:769] Sending connection context
0811 18:54:58.143472 (+ 175us) client_negotiation.cc:240] Negotiation successful
0811 18:54:58.143710 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":973,"thread_start_us":707,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.180.130@KRBTEST.COM" created.
I20260811 18:54:58.146059 28840 heartbeater.cc:347] Connected to a master server at 127.27.180.190:43381
I20260811 18:54:58.146284 28840 heartbeater.cc:464] Registering TS with master...
I20260811 18:54:58.146829 28840 heartbeater.cc:511] Master 127.27.180.190:43381 requested a full tablet report, sending...
I20260811 18:54:58.149012 28462 ts_manager.cc:194] Registered new tserver with Master: 940df53837ca4218bac0550cbcdfc982 (127.27.180.129:42007)
I20260811 18:54:58.150517 28462 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.180.129@KRBTEST.COM'} at 127.27.180.129:36463
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.180.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.130.keytab.
Entry for principal kudu/127.27.180.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.130.keytab.
I20260811 18:54:58.161839 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474496157106-28370-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474496157106-28370-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474496157106-28370-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474496157106-28370-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.180.130:0
--local_ip_for_outbound_sockets=127.27.180.130
--webserver_interface=127.27.180.130
--webserver_port=0
--tserver_master_addrs=127.27.180.190:43381
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.130.keytab
--principal=kudu/127.27.180.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:44859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:54:58.319329 28847 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:54:58.319581 28847 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:54:58.319623 28847 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:54:58.323568 28847 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:54:58.323622 28847 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:54:58.323678 28847 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.180.130
I20260811 18:54:58.328773 28847 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:44859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474496157106-28370-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474496157106-28370-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.130.keytab
--principal=kudu/127.27.180.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.180.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474496157106-28370-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.180.130
--webserver_port=0
--tserver_master_addrs=127.27.180.190:43381
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28847
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.180.130
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474496157106-28370-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:54:58.329926 28847 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:54:58.330896 28847 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:54:58.340195 28857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:54:58.340189 28856 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:54:58.340451 28847 server_base.cc:1034] running on GCE node
W20260811 18:54:58.340188 28859 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:54:58.340904 28847 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:54:58.341536 28847 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:54:58.342705 28847 hybrid_clock.cc:648] HybridClock initialized: now 1786474498342690 us; error 35 us; skew 500 ppm
Aug 11 18:54:58 dist-test-slave-2j7r krb5kdc[28383](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474498, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:54:58.345463 28847 init.cc:377] Logged in from keytab as kudu/127.27.180.130@KRBTEST.COM (short username kudu)
I20260811 18:54:58.346325 28847 webserver.cc:460] Webserver started at http://127.27.180.130:41693/ using document root <none> and password file <none>
I20260811 18:54:58.346848 28847 fs_manager.cc:359] Metadata directory not provided
I20260811 18:54:58.346921 28847 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:54:58.347124 28847 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:54:58.349181 28847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474496157106-28370-0/minicluster-data/ts-1/data/instance:
uuid: "cf966cb337c74c1194439788de6cd405"
format_stamp: "Formatted at 2026-08-11 18:54:58 on dist-test-slave-2j7r"
server_key: "e8f9767cabd8c3cebe405eed61b6ef93"
server_key_iv: "6a9914212933e718a50c6b8c71b9af53"
server_key_version: "encryptionkey@0"
I20260811 18:54:58.349723 28847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474496157106-28370-0/minicluster-data/ts-1/wal/instance:
uuid: "cf966cb337c74c1194439788de6cd405"
format_stamp: "Formatted at 2026-08-11 18:54:58 on dist-test-slave-2j7r"
server_key: "e8f9767cabd8c3cebe405eed61b6ef93"
server_key_iv: "6a9914212933e718a50c6b8c71b9af53"
server_key_version: "encryptionkey@0"
I20260811 18:54:58.353243 28847 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260811 18:54:58.355506 28871 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:54:58.356657 28847 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 18:54:58.356763 28847 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474496157106-28370-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474496157106-28370-0/minicluster-data/ts-1/wal
uuid: "cf966cb337c74c1194439788de6cd405"
format_stamp: "Formatted at 2026-08-11 18:54:58 on dist-test-slave-2j7r"
server_key: "e8f9767cabd8c3cebe405eed61b6ef93"
server_key_iv: "6a9914212933e718a50c6b8c71b9af53"
server_key_version: "encryptionkey@0"
I20260811 18:54:58.356853 28847 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474496157106-28370-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474496157106-28370-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474496157106-28370-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:54:58.386253 28847 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:54:58.388857 28847 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:54:58.389045 28847 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:54:58.389623 28847 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:54:58.390424 28847 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:54:58.390471 28847 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:54:58.390525 28847 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:54:58.390555 28847 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:54:58.399755 28847 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.130:37953
I20260811 18:54:58.399786 29087 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.130:37953 every 8 connection(s)
I20260811 18:54:58.400579 28847 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474496157106-28370-0/minicluster-data/ts-1/data/info.pb
I20260811 18:54:58.409236 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 28847
I20260811 18:54:58.409344 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474496157106-28370-0/minicluster-data/ts-1/wal/instance
Aug 11 18:54:58 dist-test-slave-2j7r krb5kdc[28383](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474498, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.130@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM
I20260811 18:54:58.409617 28370 external_mini_cluster.cc:1428] Setting key c2d35c5681f2e9e4946a74c74b9cc5b9
I20260811 18:54:58.413833 28531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:58.402345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.130:51461 (local address 127.27.180.190:43381)
0811 18:54:58.402473 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:58.402477 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:58.403533 (+ 1056us) server_negotiation.cc:407] Connection header received
0811 18:54:58.404403 (+ 870us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:58.404406 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:58.404464 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:58.404539 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:58.405699 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:58.406235 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:58.407362 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:58.407598 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:58.407687 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:58.410251 (+ 2564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:54:58.410254 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:54:58.410256 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:54:58.410282 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:54:58.411575 (+ 1293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:58.412153 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:58.412156 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:58.412158 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:58.412242 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:58.412628 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:58.412630 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:58.412632 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:58.412817 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:54:58.412895 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:58.413540 (+ 645us) server_negotiation.cc:299] Negotiation successful
0811 18:54:58.413675 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 18:54:58.414582 29095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:58.402678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:43381 (local address 127.27.180.130:51461)
0811 18:54:58.403378 (+ 700us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:58.403420 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 18:54:58.404189 (+ 769us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:54:58.404715 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:54:58.404721 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:54:58.405062 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:54:58.405548 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:58.405561 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:58.406365 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:58.406368 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:58.407230 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:58.407240 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:58.407745 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:58.407750 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:58.407924 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:58.408394 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:54:58.408409 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:54:58.410081 (+ 1672us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:54:58.411692 (+ 1611us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:54:58.411698 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:54:58.411712 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:54:58.412033 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:54:58.412361 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:54:58.412364 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:54:58.412366 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:54:58.412536 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:54:58.412913 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:54:58.412920 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:54:58.413276 (+ 356us) client_negotiation.cc:769] Sending connection context
0811 18:54:58.413451 (+ 175us) client_negotiation.cc:240] Negotiation successful
0811 18:54:58.413703 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":115,"threads_started":1}
I20260811 18:54:58.415683 29090 heartbeater.cc:347] Connected to a master server at 127.27.180.190:43381
I20260811 18:54:58.415927 29090 heartbeater.cc:464] Registering TS with master...
I20260811 18:54:58.416431 29090 heartbeater.cc:511] Master 127.27.180.190:43381 requested a full tablet report, sending...
I20260811 18:54:58.417524 28462 ts_manager.cc:194] Registered new tserver with Master: cf966cb337c74c1194439788de6cd405 (127.27.180.130:37953)
I20260811 18:54:58.418100 28462 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.180.130@KRBTEST.COM'} at 127.27.180.130:51461
WARNING: no policy specified for kudu/127.27.180.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.180.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.180.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.131.keytab.
Entry for principal kudu/127.27.180.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.131.keytab.
I20260811 18:54:58.436256 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474496157106-28370-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474496157106-28370-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474496157106-28370-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474496157106-28370-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.180.131:0
--local_ip_for_outbound_sockets=127.27.180.131
--webserver_interface=127.27.180.131
--webserver_port=0
--tserver_master_addrs=127.27.180.190:43381
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.131.keytab
--principal=kudu/127.27.180.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:44859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:54:58.548481 29100 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:54:58.548743 29100 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:54:58.548787 29100 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:54:58.552965 29100 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:54:58.553030 29100 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:54:58.553123 29100 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.180.131
I20260811 18:54:58.558308 29100 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:44859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474496157106-28370-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474496157106-28370-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.131.keytab
--principal=kudu/127.27.180.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.180.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474496157106-28370-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.180.131
--webserver_port=0
--tserver_master_addrs=127.27.180.190:43381
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29100
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.180.131
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474496157106-28370-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:54:58.559286 29100 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:54:58.560278 29100 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:54:58.569954 29110 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:54:58.569991 29114 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:54:58.569967 29112 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:54:58.570240 29100 server_base.cc:1034] running on GCE node
I20260811 18:54:58.570629 29100 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:54:58.571254 29100 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:54:58.572428 29100 hybrid_clock.cc:648] HybridClock initialized: now 1786474498572407 us; error 29 us; skew 500 ppm
Aug 11 18:54:58 dist-test-slave-2j7r krb5kdc[28383](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474498, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:54:58.575165 29100 init.cc:377] Logged in from keytab as kudu/127.27.180.131@KRBTEST.COM (short username kudu)
I20260811 18:54:58.576041 29100 webserver.cc:460] Webserver started at http://127.27.180.131:35149/ using document root <none> and password file <none>
I20260811 18:54:58.576555 29100 fs_manager.cc:359] Metadata directory not provided
I20260811 18:54:58.576624 29100 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:54:58.576829 29100 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:54:58.578830 29100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474496157106-28370-0/minicluster-data/ts-2/data/instance:
uuid: "e4795c19484c4d5ab2b8273799a23a30"
format_stamp: "Formatted at 2026-08-11 18:54:58 on dist-test-slave-2j7r"
server_key: "762d4f506c3b046abddf8946f5f9fdde"
server_key_iv: "0d3f028a55c7ea580edd3342fbb9e2bf"
server_key_version: "encryptionkey@0"
I20260811 18:54:58.579392 29100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474496157106-28370-0/minicluster-data/ts-2/wal/instance:
uuid: "e4795c19484c4d5ab2b8273799a23a30"
format_stamp: "Formatted at 2026-08-11 18:54:58 on dist-test-slave-2j7r"
server_key: "762d4f506c3b046abddf8946f5f9fdde"
server_key_iv: "0d3f028a55c7ea580edd3342fbb9e2bf"
server_key_version: "encryptionkey@0"
I20260811 18:54:58.583009 29100 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 18:54:58.585363 29125 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:54:58.586514 29100 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 18:54:58.586627 29100 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474496157106-28370-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474496157106-28370-0/minicluster-data/ts-2/wal
uuid: "e4795c19484c4d5ab2b8273799a23a30"
format_stamp: "Formatted at 2026-08-11 18:54:58 on dist-test-slave-2j7r"
server_key: "762d4f506c3b046abddf8946f5f9fdde"
server_key_iv: "0d3f028a55c7ea580edd3342fbb9e2bf"
server_key_version: "encryptionkey@0"
I20260811 18:54:58.586730 29100 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474496157106-28370-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474496157106-28370-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474496157106-28370-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:54:58.597456 29100 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:54:58.600062 29100 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:54:58.600250 29100 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:54:58.600801 29100 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:54:58.601671 29100 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:54:58.601723 29100 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:54:58.601779 29100 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:54:58.601814 29100 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:54:58.613343 29100 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.131:44843
I20260811 18:54:58.613390 29335 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.131:44843 every 8 connection(s)
I20260811 18:54:58.614209 29100 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474496157106-28370-0/minicluster-data/ts-2/data/info.pb
I20260811 18:54:58.621969 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 29100
I20260811 18:54:58.622092 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786474496157106-28370-0/minicluster-data/ts-2/wal/instance
I20260811 18:54:58.622386 28370 external_mini_cluster.cc:1428] Setting key 5c07657a46112e4097f5a36cdfd3d7f4
Aug 11 18:54:58 dist-test-slave-2j7r krb5kdc[28383](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474498, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.131@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM
I20260811 18:54:58.627418 28531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:58.616033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.131:42379 (local address 127.27.180.190:43381)
0811 18:54:58.616211 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:58.616217 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:58.617353 (+ 1136us) server_negotiation.cc:407] Connection header received
0811 18:54:58.618284 (+ 931us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:58.618287 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:58.618340 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:58.618422 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:58.619602 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:58.620133 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:58.621306 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:58.621529 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:58.621612 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:58.624119 (+ 2507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:54:58.624123 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:54:58.624124 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:54:58.624149 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:54:58.625492 (+ 1343us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:58.625973 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:58.625977 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:58.625978 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:58.626053 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:58.626387 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:58.626391 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:58.626392 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:58.626563 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:54:58.626651 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:58.627133 (+ 482us) server_negotiation.cc:299] Negotiation successful
0811 18:54:58.627263 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 18:54:58.628120 29343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:58.616440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:43381 (local address 127.27.180.131:42379)
0811 18:54:58.617197 (+ 757us) negotiation.cc:107] Waiting for socket to connect
0811 18:54:58.617238 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 18:54:58.618081 (+ 843us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:54:58.618559 (+ 478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:54:58.618565 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:54:58.618953 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:54:58.619445 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:58.619459 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:58.620250 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:58.620253 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:58.621176 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:54:58.621186 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:54:58.621644 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:54:58.621646 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:54:58.621786 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:58.622374 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:54:58.622389 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:54:58.623966 (+ 1577us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:54:58.625601 (+ 1635us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:54:58.625606 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:54:58.625619 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:54:58.625869 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:54:58.626139 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:54:58.626142 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:54:58.626144 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:54:58.626287 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:54:58.626668 (+ 381us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:54:58.626675 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:54:58.626909 (+ 234us) client_negotiation.cc:769] Sending connection context
0811 18:54:58.627067 (+ 158us) client_negotiation.cc:240] Negotiation successful
0811 18:54:58.627362 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":123,"threads_started":1}
I20260811 18:54:58.629119 29338 heartbeater.cc:347] Connected to a master server at 127.27.180.190:43381
I20260811 18:54:58.629349 29338 heartbeater.cc:464] Registering TS with master...
I20260811 18:54:58.629783 29338 heartbeater.cc:511] Master 127.27.180.190:43381 requested a full tablet report, sending...
I20260811 18:54:58.630748 28462 ts_manager.cc:194] Registered new tserver with Master: e4795c19484c4d5ab2b8273799a23a30 (127.27.180.131:44843)
I20260811 18:54:58.631275 28462 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.180.131@KRBTEST.COM'} at 127.27.180.131:42379
I20260811 18:54:58.636960 28370 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 18:54:58 dist-test-slave-2j7r krb5kdc[28383](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474498, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 18:54:58 dist-test-slave-2j7r krb5kdc[28383](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474498, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.27.180.129@KRBTEST.COM
I20260811 18:54:58.663131 29358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:58.654271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44328 (local address 127.27.180.129:42007)
0811 18:54:58.654560 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:58.654566 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:58.654591 (+ 25us) server_negotiation.cc:407] Connection header received
0811 18:54:58.654639 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:58.654644 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:58.654829 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:58.654942 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:58.655617 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:58.656411 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:58.657315 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:58.657677 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:58.657772 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:58.660071 (+ 2299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:54:58.660081 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:54:58.660094 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:54:58.660118 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:54:58.661541 (+ 1423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:58.661934 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:58.661938 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:58.661940 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:58.662021 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:58.662264 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:58.662268 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:58.662269 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:58.662517 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:54:58.662679 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:58.662815 (+ 136us) server_negotiation.cc:299] Negotiation successful
0811 18:54:58.662953 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":89,"threads_started":1}
W20260811 18:54:58.664111 28654 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44328
Aug 11 18:54:58 dist-test-slave-2j7r krb5kdc[28383](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474498, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 18:54:58 dist-test-slave-2j7r krb5kdc[28383](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474498, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.180.129@KRBTEST.COM
I20260811 18:54:58.689045 29358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:54:58.681441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44344 (local address 127.27.180.129:42007)
0811 18:54:58.681590 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0811 18:54:58.681595 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:54:58.681691 (+ 96us) server_negotiation.cc:407] Connection header received
0811 18:54:58.681828 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:54:58.681832 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:54:58.681879 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:54:58.681946 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:54:58.682630 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:58.683133 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:58.683925 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:54:58.684144 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:54:58.684230 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:54:58.686454 (+ 2224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:54:58.686457 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:54:58.686458 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:54:58.686481 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:54:58.687577 (+ 1096us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:58.687947 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:58.687951 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:58.687952 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:58.688024 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:54:58.688268 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:54:58.688272 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:54:58.688274 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:54:58.688428 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:54:58.688543 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 18:54:58.688786 (+ 243us) server_negotiation.cc:299] Negotiation successful
0811 18:54:58.688896 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 18:54:58.690112 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 28577
I20260811 18:54:58.711372 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 28847
I20260811 18:54:58.727865 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 29100
I20260811 18:54:58.741304 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 28408
2026-08-11T18:54:58Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2546 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474496157106-28370-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:54:58 dist-test-slave-2j7r krb5kdc[29372](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:54:58 dist-test-slave-2j7r krb5kdc[29372](info): set up 2 sockets
Aug 11 18:54:58 dist-test-slave-2j7r krb5kdc[29372](info): commencing operation
krb5kdc: starting...
W20260811 18:54:59.933842 28370 mini_kdc.cc:121] Time spent starting KDC: real 1.166s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474496157106-28370-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474496157106-28370-0/krb5kdc/test-user.keytab.
Aug 11 18:55:00 dist-test-slave-2j7r krb5kdc[29372](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474500, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:55:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:55:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.180.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.180.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.180.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab.
Entry for principal kudu/127.27.180.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab.
I20260811 18:55:00.056901 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474496157106-28370-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474496157106-28370-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474496157106-28370-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.190:37767
--webserver_interface=127.27.180.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab
--principal=kudu/127.27.180.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:33541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.180.190:37767
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:55:00.165866 29392 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:55:00.166128 29392 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:55:00.166165 29392 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:55:00.170007 29392 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:55:00.170053 29392 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:55:00.170069 29392 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:55:00.170084 29392 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:55:00.170099 29392 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:55:00.175184 29392 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:33541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474496157106-28370-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474496157106-28370-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.180.190:37767
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab
--principal=kudu/127.27.180.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.190:37767
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.180.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29392
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474496157106-28370-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:55:00.176122 29392 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:55:00.177066 29392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:55:00.185235 29401 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:00.185309 29400 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:55:00.185292 29392 server_base.cc:1034] running on GCE node
W20260811 18:55:00.185242 29405 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:55:00.185949 29392 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:55:00.186954 29392 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:55:00.188117 29392 hybrid_clock.cc:648] HybridClock initialized: now 1786474500188100 us; error 28 us; skew 500 ppm
Aug 11 18:55:00 dist-test-slave-2j7r krb5kdc[29372](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474500, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:55:00.190727 29392 init.cc:377] Logged in from keytab as kudu/127.27.180.190@KRBTEST.COM (short username kudu)
I20260811 18:55:00.191534 29392 webserver.cc:460] Webserver started at http://127.27.180.190:43925/ using document root <none> and password file <none>
I20260811 18:55:00.192037 29392 fs_manager.cc:359] Metadata directory not provided
I20260811 18:55:00.192101 29392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:55:00.192310 29392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:55:00.194342 29392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474496157106-28370-0/minicluster-data/master-0/data/instance:
uuid: "a7be0f1b0710439cba45e7ab15ae9232"
format_stamp: "Formatted at 2026-08-11 18:55:00 on dist-test-slave-2j7r"
server_key: "4f2e00d2aae56318ea81994e650e1a7b"
server_key_iv: "f7098860f9f2a1d1ae0898b5be5ed66e"
server_key_version: "encryptionkey@0"
I20260811 18:55:00.194867 29392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474496157106-28370-0/minicluster-data/master-0/wal/instance:
uuid: "a7be0f1b0710439cba45e7ab15ae9232"
format_stamp: "Formatted at 2026-08-11 18:55:00 on dist-test-slave-2j7r"
server_key: "4f2e00d2aae56318ea81994e650e1a7b"
server_key_iv: "f7098860f9f2a1d1ae0898b5be5ed66e"
server_key_version: "encryptionkey@0"
I20260811 18:55:00.198459 29392 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 18:55:00.200831 29413 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:00.202059 29392 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 18:55:00.202175 29392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474496157106-28370-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474496157106-28370-0/minicluster-data/master-0/wal
uuid: "a7be0f1b0710439cba45e7ab15ae9232"
format_stamp: "Formatted at 2026-08-11 18:55:00 on dist-test-slave-2j7r"
server_key: "4f2e00d2aae56318ea81994e650e1a7b"
server_key_iv: "f7098860f9f2a1d1ae0898b5be5ed66e"
server_key_version: "encryptionkey@0"
I20260811 18:55:00.202301 29392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474496157106-28370-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474496157106-28370-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474496157106-28370-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:55:00.220224 29392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:55:00.223272 29392 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:55:00.223445 29392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:55:00.230315 29392 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.190:37767
I20260811 18:55:00.230340 29500 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.190:37767 every 8 connection(s)
I20260811 18:55:00.231192 29392 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
I20260811 18:55:00.233191 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 29392
I20260811 18:55:00.233290 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474496157106-28370-0/minicluster-data/master-0/wal/instance
I20260811 18:55:00.233525 28370 external_mini_cluster.cc:1428] Setting key 65042af880cf4932c0abb3644f243051
I20260811 18:55:00.233891 29502 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:55:00.240346 29502 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7be0f1b0710439cba45e7ab15ae9232: Bootstrap starting.
Aug 11 18:55:00 dist-test-slave-2j7r krb5kdc[29372](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474500, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM
I20260811 18:55:00.242930 29502 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a7be0f1b0710439cba45e7ab15ae9232: Neither blocks nor log segments found. Creating new log.
I20260811 18:55:00.243602 29502 log.cc:824] T 00000000000000000000000000000000 P a7be0f1b0710439cba45e7ab15ae9232: Log is configured to *not* fsync() on all Append() calls
I20260811 18:55:00.245400 29502 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7be0f1b0710439cba45e7ab15ae9232: No bootstrap required, opened a new log
I20260811 18:55:00.247002 29507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:00.234781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45464 (local address 127.27.180.190:37767)
0811 18:55:00.235168 (+ 387us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:00.235184 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:00.235218 (+ 34us) server_negotiation.cc:407] Connection header received
0811 18:55:00.235840 (+ 622us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:00.235855 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:00.236151 (+ 296us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:00.236420 (+ 269us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:00.237334 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:00.238312 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:00.239466 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:00.239923 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:00.240036 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:00.242655 (+ 2619us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:00.242660 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:00.242671 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:00.242697 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:00.244211 (+ 1514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:00.244618 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:00.244623 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:00.244627 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:00.244732 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:00.244996 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:00.244999 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:00.245000 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:00.245396 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:00.245578 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:00.245856 (+ 278us) server_negotiation.cc:299] Negotiation successful
0811 18:55:00.246131 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":258,"thread_start_us":112,"threads_started":1}
I20260811 18:55:00.248502 29502 raft_consensus.cc:348] T 00000000000000000000000000000000 P a7be0f1b0710439cba45e7ab15ae9232 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7be0f1b0710439cba45e7ab15ae9232" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 37767 } }
I20260811 18:55:00.248752 29502 raft_consensus.cc:374] T 00000000000000000000000000000000 P a7be0f1b0710439cba45e7ab15ae9232 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:55:00.248807 29502 raft_consensus.cc:729] T 00000000000000000000000000000000 P a7be0f1b0710439cba45e7ab15ae9232 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7be0f1b0710439cba45e7ab15ae9232, State: Initialized, Role: FOLLOWER
I20260811 18:55:00.249249 29502 consensus_queue.cc:260] T 00000000000000000000000000000000 P a7be0f1b0710439cba45e7ab15ae9232 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7be0f1b0710439cba45e7ab15ae9232" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 37767 } }
I20260811 18:55:00.249428 29502 raft_consensus.cc:388] T 00000000000000000000000000000000 P a7be0f1b0710439cba45e7ab15ae9232 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:55:00.249498 29502 raft_consensus.cc:482] T 00000000000000000000000000000000 P a7be0f1b0710439cba45e7ab15ae9232 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:55:00.249591 29502 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a7be0f1b0710439cba45e7ab15ae9232 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:55:00.250344 29502 raft_consensus.cc:504] T 00000000000000000000000000000000 P a7be0f1b0710439cba45e7ab15ae9232 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7be0f1b0710439cba45e7ab15ae9232" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 37767 } }
I20260811 18:55:00.250644 29502 leader_election.cc:302] T 00000000000000000000000000000000 P a7be0f1b0710439cba45e7ab15ae9232 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a7be0f1b0710439cba45e7ab15ae9232; no voters:
I20260811 18:55:00.250968 29502 leader_election.cc:288] T 00000000000000000000000000000000 P a7be0f1b0710439cba45e7ab15ae9232 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:55:00.251042 29511 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a7be0f1b0710439cba45e7ab15ae9232 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:55:00.251225 29511 raft_consensus.cc:686] T 00000000000000000000000000000000 P a7be0f1b0710439cba45e7ab15ae9232 [term 1 LEADER]: Becoming Leader. State: Replica: a7be0f1b0710439cba45e7ab15ae9232, State: Running, Role: LEADER
I20260811 18:55:00.251509 29511 consensus_queue.cc:237] T 00000000000000000000000000000000 P a7be0f1b0710439cba45e7ab15ae9232 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7be0f1b0710439cba45e7ab15ae9232" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 37767 } }
I20260811 18:55:00.252173 29502 sys_catalog.cc:564] T 00000000000000000000000000000000 P a7be0f1b0710439cba45e7ab15ae9232 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:55:00.253248 29513 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7be0f1b0710439cba45e7ab15ae9232 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a7be0f1b0710439cba45e7ab15ae9232. Latest consensus state: current_term: 1 leader_uuid: "a7be0f1b0710439cba45e7ab15ae9232" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7be0f1b0710439cba45e7ab15ae9232" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 37767 } } }
I20260811 18:55:00.253221 29512 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7be0f1b0710439cba45e7ab15ae9232 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a7be0f1b0710439cba45e7ab15ae9232" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7be0f1b0710439cba45e7ab15ae9232" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 37767 } } }
I20260811 18:55:00.253391 29513 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7be0f1b0710439cba45e7ab15ae9232 [sys.catalog]: This master's current role is: LEADER
I20260811 18:55:00.253422 29512 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7be0f1b0710439cba45e7ab15ae9232 [sys.catalog]: This master's current role is: LEADER
I20260811 18:55:00.253772 29523 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:55:00.256561 29523 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:55:00.262328 29523 catalog_manager.cc:1347] Generated new cluster ID: 3abcfd1a8de944d49762d816caa7265f
I20260811 18:55:00.262392 29523 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:55:00.296525 29523 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:55:00.297497 29523 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:55:00.303231 29523 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a7be0f1b0710439cba45e7ab15ae9232: Generated new TSK 0
I20260811 18:55:00.303880 29523 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.180.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.180.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.180.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.129.keytab.
Entry for principal kudu/127.27.180.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.129.keytab.
I20260811 18:55:00.335752 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474496157106-28370-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474496157106-28370-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474496157106-28370-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474496157106-28370-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.180.129:0
--local_ip_for_outbound_sockets=127.27.180.129
--webserver_interface=127.27.180.129
--webserver_port=0
--tserver_master_addrs=127.27.180.190:37767
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.129.keytab
--principal=kudu/127.27.180.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:33541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:55:00.446097 29546 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:55:00.446328 29546 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:55:00.446379 29546 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:55:00.450381 29546 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:55:00.450438 29546 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:55:00.450515 29546 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.180.129
I20260811 18:55:00.456044 29546 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:33541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474496157106-28370-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474496157106-28370-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.129.keytab
--principal=kudu/127.27.180.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.180.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474496157106-28370-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.180.129
--webserver_port=0
--tserver_master_addrs=127.27.180.190:37767
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29546
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.180.129
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474496157106-28370-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:55:00.456992 29546 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:55:00.457975 29546 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:55:00.467135 29558 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:00.467173 29554 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:00.467149 29555 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:55:00.467356 29546 server_base.cc:1034] running on GCE node
I20260811 18:55:00.467763 29546 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:55:00.468370 29546 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:55:00.469538 29546 hybrid_clock.cc:648] HybridClock initialized: now 1786474500469515 us; error 34 us; skew 500 ppm
Aug 11 18:55:00 dist-test-slave-2j7r krb5kdc[29372](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474500, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:55:00.472164 29546 init.cc:377] Logged in from keytab as kudu/127.27.180.129@KRBTEST.COM (short username kudu)
I20260811 18:55:00.473011 29546 webserver.cc:460] Webserver started at http://127.27.180.129:40999/ using document root <none> and password file <none>
I20260811 18:55:00.473573 29546 fs_manager.cc:359] Metadata directory not provided
I20260811 18:55:00.473640 29546 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:55:00.473829 29546 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:55:00.476039 29546 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474496157106-28370-0/minicluster-data/ts-0/data/instance:
uuid: "b51b2678046242b3843c458d06c1d57b"
format_stamp: "Formatted at 2026-08-11 18:55:00 on dist-test-slave-2j7r"
server_key: "97e046020ab40b1b50b63312407b8f87"
server_key_iv: "3b41daef9d76ff9b4215f780e64b6a0f"
server_key_version: "encryptionkey@0"
I20260811 18:55:00.476574 29546 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474496157106-28370-0/minicluster-data/ts-0/wal/instance:
uuid: "b51b2678046242b3843c458d06c1d57b"
format_stamp: "Formatted at 2026-08-11 18:55:00 on dist-test-slave-2j7r"
server_key: "97e046020ab40b1b50b63312407b8f87"
server_key_iv: "3b41daef9d76ff9b4215f780e64b6a0f"
server_key_version: "encryptionkey@0"
I20260811 18:55:00.480206 29546 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260811 18:55:00.482586 29574 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:00.483796 29546 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 18:55:00.483922 29546 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474496157106-28370-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474496157106-28370-0/minicluster-data/ts-0/wal
uuid: "b51b2678046242b3843c458d06c1d57b"
format_stamp: "Formatted at 2026-08-11 18:55:00 on dist-test-slave-2j7r"
server_key: "97e046020ab40b1b50b63312407b8f87"
server_key_iv: "3b41daef9d76ff9b4215f780e64b6a0f"
server_key_version: "encryptionkey@0"
I20260811 18:55:00.484012 29546 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474496157106-28370-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474496157106-28370-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474496157106-28370-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:55:00.495154 29546 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:55:00.497789 29546 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:55:00.497967 29546 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:55:00.498517 29546 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:55:00.499311 29546 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:55:00.499359 29546 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:00.499409 29546 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:55:00.499435 29546 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:00.508179 29546 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.129:37561
I20260811 18:55:00.508241 29791 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.129:37561 every 8 connection(s)
I20260811 18:55:00.509058 29546 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474496157106-28370-0/minicluster-data/ts-0/data/info.pb
I20260811 18:55:00.511480 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 29546
I20260811 18:55:00.511588 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474496157106-28370-0/minicluster-data/ts-0/wal/instance
I20260811 18:55:00.511873 28370 external_mini_cluster.cc:1428] Setting key bdca6c28209e21317a9c19386a51a5ad
Aug 11 18:55:00 dist-test-slave-2j7r krb5kdc[29372](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474500, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.129@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.27.180.130@KRBTEST.COM; defaulting to no policy
I20260811 18:55:00.523595 29507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:00.510917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.129:56007 (local address 127.27.180.190:37767)
0811 18:55:00.511072 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:00.511076 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:00.511944 (+ 868us) server_negotiation.cc:407] Connection header received
0811 18:55:00.513631 (+ 1687us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:00.513636 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:00.513689 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:00.513795 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:00.514895 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:00.515550 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:00.517015 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:00.517355 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:00.517455 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:00.520112 (+ 2657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:00.520115 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:00.520117 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:00.520144 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:00.521502 (+ 1358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:00.522085 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:00.522088 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:00.522089 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:00.522165 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:00.522546 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:00.522549 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:00.522550 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:00.522745 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:00.522830 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:00.523299 (+ 469us) server_negotiation.cc:299] Negotiation successful
0811 18:55:00.523430 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 18:55:00.524317 29794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:00.511272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:37767 (local address 127.27.180.129:56007)
0811 18:55:00.511805 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:00.511850 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:00.513412 (+ 1562us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:00.513927 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:00.513933 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:00.514269 (+ 336us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:00.514746 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:00.514759 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:00.515686 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:00.515691 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:00.516880 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:00.516891 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:00.517485 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:00.517488 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:00.517632 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:00.518210 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:00.518226 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:00.519920 (+ 1694us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:55:00.521627 (+ 1707us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:00.521633 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:00.521648 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:00.521996 (+ 348us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:00.522261 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:00.522265 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:00.522267 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:00.522446 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:00.522835 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:55:00.522841 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:55:00.523056 (+ 215us) client_negotiation.cc:769] Sending connection context
0811 18:55:00.523232 (+ 176us) client_negotiation.cc:240] Negotiation successful
0811 18:55:00.523536 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":133,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.180.130@KRBTEST.COM" created.
I20260811 18:55:00.525620 29792 heartbeater.cc:347] Connected to a master server at 127.27.180.190:37767
I20260811 18:55:00.525851 29792 heartbeater.cc:464] Registering TS with master...
I20260811 18:55:00.526362 29792 heartbeater.cc:511] Master 127.27.180.190:37767 requested a full tablet report, sending...
I20260811 18:55:00.527940 29439 ts_manager.cc:194] Registered new tserver with Master: b51b2678046242b3843c458d06c1d57b (127.27.180.129:37561)
I20260811 18:55:00.529114 29439 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.180.129@KRBTEST.COM'} at 127.27.180.129:56007
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.180.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.130.keytab.
Entry for principal kudu/127.27.180.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.130.keytab.
I20260811 18:55:00.541927 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474496157106-28370-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474496157106-28370-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474496157106-28370-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474496157106-28370-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.180.130:0
--local_ip_for_outbound_sockets=127.27.180.130
--webserver_interface=127.27.180.130
--webserver_port=0
--tserver_master_addrs=127.27.180.190:37767
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.130.keytab
--principal=kudu/127.27.180.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:33541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:55:00.650741 29800 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:55:00.650972 29800 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:55:00.651012 29800 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:55:00.655110 29800 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:55:00.655164 29800 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:55:00.655226 29800 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.180.130
I20260811 18:55:00.660271 29800 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:33541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474496157106-28370-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474496157106-28370-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.130.keytab
--principal=kudu/127.27.180.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.180.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474496157106-28370-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.180.130
--webserver_port=0
--tserver_master_addrs=127.27.180.190:37767
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29800
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.180.130
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474496157106-28370-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:55:00.661240 29800 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:55:00.662209 29800 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:55:00.671296 29813 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:00.671283 29817 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:00.671286 29814 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:55:00.671651 29800 server_base.cc:1034] running on GCE node
I20260811 18:55:00.672039 29800 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:55:00.672636 29800 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:55:00.673810 29800 hybrid_clock.cc:648] HybridClock initialized: now 1786474500673785 us; error 37 us; skew 500 ppm
Aug 11 18:55:00 dist-test-slave-2j7r krb5kdc[29372](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474500, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:55:00.676486 29800 init.cc:377] Logged in from keytab as kudu/127.27.180.130@KRBTEST.COM (short username kudu)
I20260811 18:55:00.677340 29800 webserver.cc:460] Webserver started at http://127.27.180.130:45705/ using document root <none> and password file <none>
I20260811 18:55:00.677866 29800 fs_manager.cc:359] Metadata directory not provided
I20260811 18:55:00.677935 29800 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:55:00.678121 29800 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:55:00.680133 29800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474496157106-28370-0/minicluster-data/ts-1/data/instance:
uuid: "437f613592ec495090db03fb118a62b5"
format_stamp: "Formatted at 2026-08-11 18:55:00 on dist-test-slave-2j7r"
server_key: "99b39486117cec53f3dae318a54234d5"
server_key_iv: "92dd42d79496454d94643bd0e32d71e2"
server_key_version: "encryptionkey@0"
I20260811 18:55:00.680655 29800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474496157106-28370-0/minicluster-data/ts-1/wal/instance:
uuid: "437f613592ec495090db03fb118a62b5"
format_stamp: "Formatted at 2026-08-11 18:55:00 on dist-test-slave-2j7r"
server_key: "99b39486117cec53f3dae318a54234d5"
server_key_iv: "92dd42d79496454d94643bd0e32d71e2"
server_key_version: "encryptionkey@0"
I20260811 18:55:00.684180 29800 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260811 18:55:00.686488 29828 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:00.687673 29800 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 18:55:00.687786 29800 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474496157106-28370-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474496157106-28370-0/minicluster-data/ts-1/wal
uuid: "437f613592ec495090db03fb118a62b5"
format_stamp: "Formatted at 2026-08-11 18:55:00 on dist-test-slave-2j7r"
server_key: "99b39486117cec53f3dae318a54234d5"
server_key_iv: "92dd42d79496454d94643bd0e32d71e2"
server_key_version: "encryptionkey@0"
I20260811 18:55:00.687873 29800 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474496157106-28370-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474496157106-28370-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474496157106-28370-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:55:00.699534 29800 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:55:00.702147 29800 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:55:00.702309 29800 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:55:00.702850 29800 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:55:00.703622 29800 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:55:00.703668 29800 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:00.703720 29800 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:55:00.703748 29800 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:00.712129 29800 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.130:42017
I20260811 18:55:00.712188 30050 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.130:42017 every 8 connection(s)
I20260811 18:55:00.712934 29800 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474496157106-28370-0/minicluster-data/ts-1/data/info.pb
I20260811 18:55:00.717654 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 29800
I20260811 18:55:00.717788 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474496157106-28370-0/minicluster-data/ts-1/wal/instance
I20260811 18:55:00.718070 28370 external_mini_cluster.cc:1428] Setting key b399beac3b56c679d9f0c9328f681eff
Aug 11 18:55:00 dist-test-slave-2j7r krb5kdc[29372](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474500, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.130@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM
I20260811 18:55:00.726876 29507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:00.714645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.130:47671 (local address 127.27.180.190:37767)
0811 18:55:00.714786 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:00.714791 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:00.715621 (+ 830us) server_negotiation.cc:407] Connection header received
0811 18:55:00.716778 (+ 1157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:00.716781 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:00.716828 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:00.716901 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:00.718265 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:00.718856 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:00.720295 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:00.720517 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:00.720594 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:00.723111 (+ 2517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:00.723114 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:00.723116 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:00.723140 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:00.724569 (+ 1429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:00.725178 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:00.725181 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:00.725182 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:00.725287 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:00.725692 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:00.725696 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:00.725710 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:00.725931 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:00.726008 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:00.726633 (+ 625us) server_negotiation.cc:299] Negotiation successful
0811 18:55:00.726750 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 18:55:00.728087 30054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:00.715041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:37767 (local address 127.27.180.130:47671)
0811 18:55:00.715475 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:00.715515 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:00.716606 (+ 1091us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:00.717039 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:00.717046 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:00.717487 (+ 441us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:00.718120 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:00.718141 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:00.718983 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:00.718986 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:00.720175 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:00.720187 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:00.720653 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:00.720657 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:00.720848 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:00.721408 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:00.721429 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:00.722907 (+ 1478us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:55:00.724694 (+ 1787us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:00.724700 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:00.724717 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:00.725062 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:00.725380 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:00.725383 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:00.725386 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:00.725571 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:00.726038 (+ 467us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:55:00.726045 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:55:00.726376 (+ 331us) client_negotiation.cc:769] Sending connection context
0811 18:55:00.726579 (+ 203us) client_negotiation.cc:240] Negotiation successful
0811 18:55:00.726918 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":106,"threads_started":1}
WARNING: no policy specified for kudu/127.27.180.131@KRBTEST.COM; defaulting to no policy
I20260811 18:55:00.729185 30051 heartbeater.cc:347] Connected to a master server at 127.27.180.190:37767
I20260811 18:55:00.729395 30051 heartbeater.cc:464] Registering TS with master...
I20260811 18:55:00.729864 30051 heartbeater.cc:511] Master 127.27.180.190:37767 requested a full tablet report, sending...
I20260811 18:55:00.730897 29439 ts_manager.cc:194] Registered new tserver with Master: 437f613592ec495090db03fb118a62b5 (127.27.180.130:42017)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.180.131@KRBTEST.COM" created.
I20260811 18:55:00.731628 29439 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.180.130@KRBTEST.COM'} at 127.27.180.130:47671
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.180.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.131.keytab.
Entry for principal kudu/127.27.180.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.131.keytab.
I20260811 18:55:00.748337 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474496157106-28370-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474496157106-28370-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474496157106-28370-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474496157106-28370-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.180.131:0
--local_ip_for_outbound_sockets=127.27.180.131
--webserver_interface=127.27.180.131
--webserver_port=0
--tserver_master_addrs=127.27.180.190:37767
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.131.keytab
--principal=kudu/127.27.180.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:33541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:55:00.857340 30063 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:55:00.857563 30063 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:55:00.857602 30063 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:55:00.861519 30063 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:55:00.861578 30063 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:55:00.861655 30063 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.180.131
I20260811 18:55:00.868072 30063 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:33541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474496157106-28370-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474496157106-28370-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.131.keytab
--principal=kudu/127.27.180.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.180.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474496157106-28370-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.180.131
--webserver_port=0
--tserver_master_addrs=127.27.180.190:37767
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30063
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.180.131
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474496157106-28370-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:55:00.869267 30063 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:55:00.870399 30063 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:55:00.879782 30080 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:00.879866 30076 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:00.879838 30073 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:55:00.879844 30063 server_base.cc:1034] running on GCE node
I20260811 18:55:00.880367 30063 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:55:00.880961 30063 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:55:00.882133 30063 hybrid_clock.cc:648] HybridClock initialized: now 1786474500882110 us; error 36 us; skew 500 ppm
Aug 11 18:55:00 dist-test-slave-2j7r krb5kdc[29372](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474500, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:55:00.884877 30063 init.cc:377] Logged in from keytab as kudu/127.27.180.131@KRBTEST.COM (short username kudu)
I20260811 18:55:00.885802 30063 webserver.cc:460] Webserver started at http://127.27.180.131:45123/ using document root <none> and password file <none>
I20260811 18:55:00.886399 30063 fs_manager.cc:359] Metadata directory not provided
I20260811 18:55:00.886475 30063 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:55:00.886673 30063 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:55:00.888756 30063 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474496157106-28370-0/minicluster-data/ts-2/data/instance:
uuid: "0b8de0e94c1c411a8c407e92312bd1a8"
format_stamp: "Formatted at 2026-08-11 18:55:00 on dist-test-slave-2j7r"
server_key: "693e94e921ccacbecd97b1031e687058"
server_key_iv: "483921cd90fe5ecc390d3e15fdaac4be"
server_key_version: "encryptionkey@0"
I20260811 18:55:00.889374 30063 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474496157106-28370-0/minicluster-data/ts-2/wal/instance:
uuid: "0b8de0e94c1c411a8c407e92312bd1a8"
format_stamp: "Formatted at 2026-08-11 18:55:00 on dist-test-slave-2j7r"
server_key: "693e94e921ccacbecd97b1031e687058"
server_key_iv: "483921cd90fe5ecc390d3e15fdaac4be"
server_key_version: "encryptionkey@0"
I20260811 18:55:00.893303 30063 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260811 18:55:00.895766 30092 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:00.896885 30063 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 18:55:00.896998 30063 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474496157106-28370-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474496157106-28370-0/minicluster-data/ts-2/wal
uuid: "0b8de0e94c1c411a8c407e92312bd1a8"
format_stamp: "Formatted at 2026-08-11 18:55:00 on dist-test-slave-2j7r"
server_key: "693e94e921ccacbecd97b1031e687058"
server_key_iv: "483921cd90fe5ecc390d3e15fdaac4be"
server_key_version: "encryptionkey@0"
I20260811 18:55:00.897135 30063 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474496157106-28370-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474496157106-28370-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474496157106-28370-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:55:00.905993 30063 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:55:00.909025 30063 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:55:00.909236 30063 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:55:00.909801 30063 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:55:00.910616 30063 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:55:00.910679 30063 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:00.910738 30063 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:55:00.910771 30063 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:00.919904 30063 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.131:36897
I20260811 18:55:00.919930 30310 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.131:36897 every 8 connection(s)
I20260811 18:55:00.920745 30063 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474496157106-28370-0/minicluster-data/ts-2/data/info.pb
I20260811 18:55:00.924420 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 30063
I20260811 18:55:00.924556 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786474496157106-28370-0/minicluster-data/ts-2/wal/instance
I20260811 18:55:00.924903 28370 external_mini_cluster.cc:1428] Setting key 4314bec30be68694e7bd9b2934425a72
Aug 11 18:55:00 dist-test-slave-2j7r krb5kdc[29372](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474500, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.131@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM
I20260811 18:55:00.933558 29507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:00.922503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.131:53905 (local address 127.27.180.190:37767)
0811 18:55:00.922695 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:00.922701 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:00.923493 (+ 792us) server_negotiation.cc:407] Connection header received
0811 18:55:00.924804 (+ 1311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:00.924807 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:00.924857 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:00.924930 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:00.926229 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:00.926796 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:00.928017 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:00.928230 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:00.928326 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:00.930734 (+ 2408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:00.930738 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:00.930740 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:00.930764 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:00.931939 (+ 1175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:00.932355 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:00.932358 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:00.932359 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:00.932431 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:00.932676 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:00.932678 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:00.932679 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:00.932835 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:00.932910 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:00.933343 (+ 433us) server_negotiation.cc:299] Negotiation successful
0811 18:55:00.933440 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 18:55:00.934268 30314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:00.922868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:37767 (local address 127.27.180.131:53905)
0811 18:55:00.923336 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:00.923376 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:00.924576 (+ 1200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:00.925157 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:00.925165 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:00.925516 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:00.926073 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:00.926092 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:00.926931 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:00.926935 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:00.927884 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:00.927895 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:00.928372 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:00.928376 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:00.928547 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:00.929059 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:00.929076 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:00.930521 (+ 1445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:55:00.932030 (+ 1509us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:00.932034 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:00.932044 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:00.932277 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:00.932497 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:00.932499 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:00.932500 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:00.932612 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:00.932906 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:55:00.932910 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:55:00.933150 (+ 240us) client_negotiation.cc:769] Sending connection context
0811 18:55:00.933310 (+ 160us) client_negotiation.cc:240] Negotiation successful
0811 18:55:00.933534 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":127,"threads_started":1}
I20260811 18:55:00.935230 30311 heartbeater.cc:347] Connected to a master server at 127.27.180.190:37767
I20260811 18:55:00.935442 30311 heartbeater.cc:464] Registering TS with master...
I20260811 18:55:00.935887 30311 heartbeater.cc:511] Master 127.27.180.190:37767 requested a full tablet report, sending...
I20260811 18:55:00.936836 29439 ts_manager.cc:194] Registered new tserver with Master: 0b8de0e94c1c411a8c407e92312bd1a8 (127.27.180.131:36897)
I20260811 18:55:00.937474 29439 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.180.131@KRBTEST.COM'} at 127.27.180.131:53905
I20260811 18:55:00.938795 28370 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 18:55:00.949666 29507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:00.943100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45466 (local address 127.27.180.190:37767)
0811 18:55:00.943209 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:00.943214 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:00.943376 (+ 162us) server_negotiation.cc:407] Connection header received
0811 18:55:00.943457 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:00.943459 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:00.943495 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:00.943557 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:00.944167 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:00.944670 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:00.945514 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:00.945717 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:00.945794 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:00.947117 (+ 1323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:00.947120 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:00.947122 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:00.947144 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:00.948313 (+ 1169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:00.948660 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:00.948662 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:00.948663 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:00.948733 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:00.948934 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:00.948936 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:00.948937 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:00.949159 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:00.949273 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:00.949458 (+ 185us) server_negotiation.cc:299] Negotiation successful
0811 18:55:00.949544 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 18:55:00.955281 29439 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45466:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:55:00.958657 29439 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:55:00.974714 30330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:00.968685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.129:37561 (local address 127.0.0.1:43854)
0811 18:55:00.969433 (+ 748us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:00.969484 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:00.969609 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:00.970917 (+ 1308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:00.970920 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:00.970936 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:55:00.971023 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:00.971029 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:00.972095 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:00.972098 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:00.973234 (+ 1136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:00.973244 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:00.974170 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:00.974174 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:00.974282 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:00.974326 (+ 44us) client_negotiation.cc:769] Sending connection context
0811 18:55:00.974443 (+ 117us) client_negotiation.cc:240] Negotiation successful
0811 18:55:00.974559 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":632,"mutex_wait_us":97,"thread_start_us":94,"threads_started":1}
I20260811 18:55:00.974892 30338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:00.969331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48248 (local address 127.27.180.130:42017)
0811 18:55:00.969893 (+ 562us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:00.969903 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:00.969922 (+ 19us) server_negotiation.cc:407] Connection header received
0811 18:55:00.969984 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:00.969988 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:00.970133 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:00.970249 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:55:00.970571 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:00.971796 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:00.973732 (+ 1936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:00.974351 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:00.974474 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:00.974592 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:00.974676 (+ 84us) server_negotiation.cc:299] Negotiation successful
0811 18:55:00.974762 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":473,"thread_start_us":153,"threads_started":1}
I20260811 18:55:00.975314 30329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:00.968531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.131:36897 (local address 127.0.0.1:49160)
0811 18:55:00.969731 (+ 1200us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:00.969745 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:00.969839 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:00.970487 (+ 648us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:00.970490 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:00.970507 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:55:00.970603 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:00.970609 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:00.971963 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:00.971967 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:00.973589 (+ 1622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:00.973601 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:00.974796 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:00.974800 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:00.974888 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:00.974908 (+ 20us) client_negotiation.cc:769] Sending connection context
0811 18:55:00.974967 (+ 59us) client_negotiation.cc:240] Negotiation successful
0811 18:55:00.975170 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":598,"mutex_wait_us":180,"thread_start_us":69,"threads_started":1}
I20260811 18:55:00.975322 30335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:00.969346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43854 (local address 127.27.180.129:37561)
0811 18:55:00.970524 (+ 1178us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:00.970535 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:00.970554 (+ 19us) server_negotiation.cc:407] Connection header received
0811 18:55:00.970623 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:00.970628 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:00.970798 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:00.970923 (+ 125us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:55:00.971109 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:00.971937 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:00.973361 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:00.974042 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:00.974165 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:00.974363 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:00.975089 (+ 726us) server_negotiation.cc:299] Negotiation successful
0811 18:55:00.975186 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1079,"thread_start_us":133,"threads_started":1}
I20260811 18:55:00.975729 30333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:00.968698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49160 (local address 127.27.180.131:36897)
0811 18:55:00.970078 (+ 1380us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:00.970087 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:00.970106 (+ 19us) server_negotiation.cc:407] Connection header received
0811 18:55:00.970162 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:00.970165 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:00.970337 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:00.970487 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:55:00.970716 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:00.971825 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:00.973717 (+ 1892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:00.974510 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:00.974620 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:00.974736 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:00.975484 (+ 748us) server_negotiation.cc:299] Negotiation successful
0811 18:55:00.975585 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1294,"thread_start_us":171,"threads_started":1}
I20260811 18:55:00.976435 30334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:00.968907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.130:42017 (local address 127.0.0.1:48248)
0811 18:55:00.969788 (+ 881us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:00.969800 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:00.969863 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:00.970285 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:00.970290 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:00.970311 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:55:00.970440 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:00.970447 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:00.971963 (+ 1516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:00.971967 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:00.973599 (+ 1632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:00.973606 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:00.974474 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:00.974477 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:00.974557 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:00.974583 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 18:55:00.974650 (+ 67us) client_negotiation.cc:240] Negotiation successful
0811 18:55:00.974719 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":828,"mutex_wait_us":126,"thread_start_us":96,"threads_started":1}
I20260811 18:55:00.978427 30175 tablet_service.cc:1467] Processing CreateTablet for tablet cff19b6da213433d8ab4bd7b044848ab (DEFAULT_TABLE table=test-table [id=d87ab848ca05473aa6791543fd85592a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:55:00.978749 29920 tablet_service.cc:1467] Processing CreateTablet for tablet cff19b6da213433d8ab4bd7b044848ab (DEFAULT_TABLE table=test-table [id=d87ab848ca05473aa6791543fd85592a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:55:00.979763 30175 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cff19b6da213433d8ab4bd7b044848ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:55:00.979887 29920 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cff19b6da213433d8ab4bd7b044848ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:55:00.979853 29665 tablet_service.cc:1467] Processing CreateTablet for tablet cff19b6da213433d8ab4bd7b044848ab (DEFAULT_TABLE table=test-table [id=d87ab848ca05473aa6791543fd85592a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:55:00.980799 29665 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cff19b6da213433d8ab4bd7b044848ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:55:00.986397 30339 tablet_bootstrap.cc:492] T cff19b6da213433d8ab4bd7b044848ab P 0b8de0e94c1c411a8c407e92312bd1a8: Bootstrap starting.
I20260811 18:55:00.987797 30341 tablet_bootstrap.cc:492] T cff19b6da213433d8ab4bd7b044848ab P b51b2678046242b3843c458d06c1d57b: Bootstrap starting.
I20260811 18:55:00.988349 30340 tablet_bootstrap.cc:492] T cff19b6da213433d8ab4bd7b044848ab P 437f613592ec495090db03fb118a62b5: Bootstrap starting.
I20260811 18:55:00.989645 30339 tablet_bootstrap.cc:654] T cff19b6da213433d8ab4bd7b044848ab P 0b8de0e94c1c411a8c407e92312bd1a8: Neither blocks nor log segments found. Creating new log.
I20260811 18:55:00.990276 30341 tablet_bootstrap.cc:654] T cff19b6da213433d8ab4bd7b044848ab P b51b2678046242b3843c458d06c1d57b: Neither blocks nor log segments found. Creating new log.
I20260811 18:55:00.990833 30340 tablet_bootstrap.cc:654] T cff19b6da213433d8ab4bd7b044848ab P 437f613592ec495090db03fb118a62b5: Neither blocks nor log segments found. Creating new log.
I20260811 18:55:00.990962 30339 log.cc:824] T cff19b6da213433d8ab4bd7b044848ab P 0b8de0e94c1c411a8c407e92312bd1a8: Log is configured to *not* fsync() on all Append() calls
I20260811 18:55:00.991578 30341 log.cc:824] T cff19b6da213433d8ab4bd7b044848ab P b51b2678046242b3843c458d06c1d57b: Log is configured to *not* fsync() on all Append() calls
I20260811 18:55:00.991596 30340 log.cc:824] T cff19b6da213433d8ab4bd7b044848ab P 437f613592ec495090db03fb118a62b5: Log is configured to *not* fsync() on all Append() calls
I20260811 18:55:00.993073 30339 tablet_bootstrap.cc:492] T cff19b6da213433d8ab4bd7b044848ab P 0b8de0e94c1c411a8c407e92312bd1a8: No bootstrap required, opened a new log
I20260811 18:55:00.993305 30339 ts_tablet_manager.cc:1397] T cff19b6da213433d8ab4bd7b044848ab P 0b8de0e94c1c411a8c407e92312bd1a8: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.004s
I20260811 18:55:00.993563 30341 tablet_bootstrap.cc:492] T cff19b6da213433d8ab4bd7b044848ab P b51b2678046242b3843c458d06c1d57b: No bootstrap required, opened a new log
I20260811 18:55:00.993748 30341 ts_tablet_manager.cc:1397] T cff19b6da213433d8ab4bd7b044848ab P b51b2678046242b3843c458d06c1d57b: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 18:55:00.994841 30340 tablet_bootstrap.cc:492] T cff19b6da213433d8ab4bd7b044848ab P 437f613592ec495090db03fb118a62b5: No bootstrap required, opened a new log
I20260811 18:55:00.995012 30340 ts_tablet_manager.cc:1397] T cff19b6da213433d8ab4bd7b044848ab P 437f613592ec495090db03fb118a62b5: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260811 18:55:00.996471 30339 raft_consensus.cc:348] T cff19b6da213433d8ab4bd7b044848ab P 0b8de0e94c1c411a8c407e92312bd1a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b8de0e94c1c411a8c407e92312bd1a8" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 36897 } } peers { permanent_uuid: "b51b2678046242b3843c458d06c1d57b" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 37561 } } peers { permanent_uuid: "437f613592ec495090db03fb118a62b5" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 42017 } }
I20260811 18:55:00.996778 30339 raft_consensus.cc:374] T cff19b6da213433d8ab4bd7b044848ab P 0b8de0e94c1c411a8c407e92312bd1a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:55:00.996837 30339 raft_consensus.cc:729] T cff19b6da213433d8ab4bd7b044848ab P 0b8de0e94c1c411a8c407e92312bd1a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b8de0e94c1c411a8c407e92312bd1a8, State: Initialized, Role: FOLLOWER
I20260811 18:55:00.997299 30339 consensus_queue.cc:260] T cff19b6da213433d8ab4bd7b044848ab P 0b8de0e94c1c411a8c407e92312bd1a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b8de0e94c1c411a8c407e92312bd1a8" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 36897 } } peers { permanent_uuid: "b51b2678046242b3843c458d06c1d57b" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 37561 } } peers { permanent_uuid: "437f613592ec495090db03fb118a62b5" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 42017 } }
I20260811 18:55:00.998013 30341 raft_consensus.cc:348] T cff19b6da213433d8ab4bd7b044848ab P b51b2678046242b3843c458d06c1d57b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b8de0e94c1c411a8c407e92312bd1a8" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 36897 } } peers { permanent_uuid: "b51b2678046242b3843c458d06c1d57b" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 37561 } } peers { permanent_uuid: "437f613592ec495090db03fb118a62b5" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 42017 } }
I20260811 18:55:00.998214 30339 ts_tablet_manager.cc:1428] T cff19b6da213433d8ab4bd7b044848ab P 0b8de0e94c1c411a8c407e92312bd1a8: Time spent starting tablet: real 0.005s user 0.001s sys 0.003s
I20260811 18:55:00.998281 30341 raft_consensus.cc:374] T cff19b6da213433d8ab4bd7b044848ab P b51b2678046242b3843c458d06c1d57b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:55:00.998348 30341 raft_consensus.cc:729] T cff19b6da213433d8ab4bd7b044848ab P b51b2678046242b3843c458d06c1d57b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b51b2678046242b3843c458d06c1d57b, State: Initialized, Role: FOLLOWER
I20260811 18:55:00.998656 30311 heartbeater.cc:503] Master 127.27.180.190:37767 was elected leader, sending a full tablet report...
I20260811 18:55:00.998823 30341 consensus_queue.cc:260] T cff19b6da213433d8ab4bd7b044848ab P b51b2678046242b3843c458d06c1d57b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b8de0e94c1c411a8c407e92312bd1a8" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 36897 } } peers { permanent_uuid: "b51b2678046242b3843c458d06c1d57b" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 37561 } } peers { permanent_uuid: "437f613592ec495090db03fb118a62b5" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 42017 } }
I20260811 18:55:00.999089 30340 raft_consensus.cc:348] T cff19b6da213433d8ab4bd7b044848ab P 437f613592ec495090db03fb118a62b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b8de0e94c1c411a8c407e92312bd1a8" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 36897 } } peers { permanent_uuid: "b51b2678046242b3843c458d06c1d57b" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 37561 } } peers { permanent_uuid: "437f613592ec495090db03fb118a62b5" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 42017 } }
I20260811 18:55:00.999331 30340 raft_consensus.cc:374] T cff19b6da213433d8ab4bd7b044848ab P 437f613592ec495090db03fb118a62b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:55:00.999379 30340 raft_consensus.cc:729] T cff19b6da213433d8ab4bd7b044848ab P 437f613592ec495090db03fb118a62b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 437f613592ec495090db03fb118a62b5, State: Initialized, Role: FOLLOWER
I20260811 18:55:00.999506 29792 heartbeater.cc:503] Master 127.27.180.190:37767 was elected leader, sending a full tablet report...
I20260811 18:55:00.999825 30341 ts_tablet_manager.cc:1428] T cff19b6da213433d8ab4bd7b044848ab P b51b2678046242b3843c458d06c1d57b: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 18:55:00.999820 30340 consensus_queue.cc:260] T cff19b6da213433d8ab4bd7b044848ab P 437f613592ec495090db03fb118a62b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b8de0e94c1c411a8c407e92312bd1a8" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 36897 } } peers { permanent_uuid: "b51b2678046242b3843c458d06c1d57b" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 37561 } } peers { permanent_uuid: "437f613592ec495090db03fb118a62b5" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 42017 } }
I20260811 18:55:01.000485 30051 heartbeater.cc:503] Master 127.27.180.190:37767 was elected leader, sending a full tablet report...
I20260811 18:55:01.000849 30340 ts_tablet_manager.cc:1428] T cff19b6da213433d8ab4bd7b044848ab P 437f613592ec495090db03fb118a62b5: Time spent starting tablet: real 0.006s user 0.001s sys 0.005s
W20260811 18:55:01.010233 29793 tablet.cc:2369] T cff19b6da213433d8ab4bd7b044848ab P b51b2678046242b3843c458d06c1d57b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:55:01.150290 30348 raft_consensus.cc:482] T cff19b6da213433d8ab4bd7b044848ab P b51b2678046242b3843c458d06c1d57b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:55:01.150537 30348 raft_consensus.cc:504] T cff19b6da213433d8ab4bd7b044848ab P b51b2678046242b3843c458d06c1d57b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b8de0e94c1c411a8c407e92312bd1a8" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 36897 } } peers { permanent_uuid: "b51b2678046242b3843c458d06c1d57b" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 37561 } } peers { permanent_uuid: "437f613592ec495090db03fb118a62b5" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 42017 } }
I20260811 18:55:01.151748 30348 leader_election.cc:288] T cff19b6da213433d8ab4bd7b044848ab P b51b2678046242b3843c458d06c1d57b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0b8de0e94c1c411a8c407e92312bd1a8 (127.27.180.131:36897), 437f613592ec495090db03fb118a62b5 (127.27.180.130:42017)
I20260811 18:55:01.156770 30338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:01.152234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.129:35429 (local address 127.27.180.130:42017)
0811 18:55:01.152363 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:01.152367 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:01.152797 (+ 430us) server_negotiation.cc:407] Connection header received
0811 18:55:01.152900 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:01.152903 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:01.152953 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:01.153016 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:55:01.153334 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:01.154065 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:01.155514 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:01.156192 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:01.156294 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:01.156337 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:01.156563 (+ 226us) server_negotiation.cc:299] Negotiation successful
0811 18:55:01.156637 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 18:55:01.156769 30333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:01.152206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.129:42927 (local address 127.27.180.131:36897)
0811 18:55:01.152360 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:01.152365 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:01.152465 (+ 100us) server_negotiation.cc:407] Connection header received
0811 18:55:01.152618 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:01.152621 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:01.152665 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:01.152730 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:55:01.153122 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:01.153918 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:01.155514 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:01.156126 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:01.156268 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:01.156322 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:01.156565 (+ 243us) server_negotiation.cc:299] Negotiation successful
0811 18:55:01.156637 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 18:55:01.156801 30350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:01.152058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.131:36897 (local address 127.27.180.129:42927)
0811 18:55:01.152364 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:01.152379 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:01.152494 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:01.152800 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:01.152806 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:01.152827 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:55:01.152942 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:01.152948 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:01.154019 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:01.154023 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:01.155384 (+ 1361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:01.155394 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:01.156253 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:01.156257 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:01.156390 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:01.156457 (+ 67us) client_negotiation.cc:769] Sending connection context
0811 18:55:01.156532 (+ 75us) client_negotiation.cc:240] Negotiation successful
0811 18:55:01.156618 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221,"mutex_wait_us":68,"thread_start_us":95,"threads_started":1}
I20260811 18:55:01.156804 30358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:01.152058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.130:42017 (local address 127.27.180.129:35429)
0811 18:55:01.152651 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:01.152662 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:01.152769 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:01.153040 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:01.153042 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:01.153058 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:55:01.153192 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:01.153200 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:01.154148 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:01.154151 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:01.155383 (+ 1232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:01.155395 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:01.156303 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:01.156306 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:01.156433 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:01.156471 (+ 38us) client_negotiation.cc:769] Sending connection context
0811 18:55:01.156532 (+ 61us) client_negotiation.cc:240] Negotiation successful
0811 18:55:01.156618 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":532,"mutex_wait_us":355,"thread_start_us":113,"threads_started":1}
I20260811 18:55:01.157604 29957 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cff19b6da213433d8ab4bd7b044848ab" candidate_uuid: "b51b2678046242b3843c458d06c1d57b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "437f613592ec495090db03fb118a62b5" is_pre_election: true
I20260811 18:55:01.157601 30217 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cff19b6da213433d8ab4bd7b044848ab" candidate_uuid: "b51b2678046242b3843c458d06c1d57b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b8de0e94c1c411a8c407e92312bd1a8" is_pre_election: true
I20260811 18:55:01.158034 29957 raft_consensus.cc:2454] T cff19b6da213433d8ab4bd7b044848ab P 437f613592ec495090db03fb118a62b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b51b2678046242b3843c458d06c1d57b in term 0.
I20260811 18:55:01.158607 29578 leader_election.cc:302] T cff19b6da213433d8ab4bd7b044848ab P b51b2678046242b3843c458d06c1d57b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 437f613592ec495090db03fb118a62b5, b51b2678046242b3843c458d06c1d57b; no voters:
I20260811 18:55:01.158855 30348 raft_consensus.cc:2781] T cff19b6da213433d8ab4bd7b044848ab P b51b2678046242b3843c458d06c1d57b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:55:01.158938 30348 raft_consensus.cc:482] T cff19b6da213433d8ab4bd7b044848ab P b51b2678046242b3843c458d06c1d57b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:55:01.158970 30348 raft_consensus.cc:3037] T cff19b6da213433d8ab4bd7b044848ab P b51b2678046242b3843c458d06c1d57b [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:55:01.159866 30217 raft_consensus.cc:2454] T cff19b6da213433d8ab4bd7b044848ab P 0b8de0e94c1c411a8c407e92312bd1a8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b51b2678046242b3843c458d06c1d57b in term 0.
I20260811 18:55:01.160020 30348 raft_consensus.cc:504] T cff19b6da213433d8ab4bd7b044848ab P b51b2678046242b3843c458d06c1d57b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b8de0e94c1c411a8c407e92312bd1a8" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 36897 } } peers { permanent_uuid: "b51b2678046242b3843c458d06c1d57b" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 37561 } } peers { permanent_uuid: "437f613592ec495090db03fb118a62b5" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 42017 } }
I20260811 18:55:01.160354 30348 leader_election.cc:288] T cff19b6da213433d8ab4bd7b044848ab P b51b2678046242b3843c458d06c1d57b [CANDIDATE]: Term 1 election: Requested vote from peers 0b8de0e94c1c411a8c407e92312bd1a8 (127.27.180.131:36897), 437f613592ec495090db03fb118a62b5 (127.27.180.130:42017)
I20260811 18:55:01.160709 30217 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cff19b6da213433d8ab4bd7b044848ab" candidate_uuid: "b51b2678046242b3843c458d06c1d57b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b8de0e94c1c411a8c407e92312bd1a8"
I20260811 18:55:01.160699 29957 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cff19b6da213433d8ab4bd7b044848ab" candidate_uuid: "b51b2678046242b3843c458d06c1d57b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "437f613592ec495090db03fb118a62b5"
I20260811 18:55:01.160811 30217 raft_consensus.cc:3037] T cff19b6da213433d8ab4bd7b044848ab P 0b8de0e94c1c411a8c407e92312bd1a8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:55:01.160821 29957 raft_consensus.cc:3037] T cff19b6da213433d8ab4bd7b044848ab P 437f613592ec495090db03fb118a62b5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:55:01.161872 29957 raft_consensus.cc:2454] T cff19b6da213433d8ab4bd7b044848ab P 437f613592ec495090db03fb118a62b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b51b2678046242b3843c458d06c1d57b in term 1.
I20260811 18:55:01.161909 30217 raft_consensus.cc:2454] T cff19b6da213433d8ab4bd7b044848ab P 0b8de0e94c1c411a8c407e92312bd1a8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b51b2678046242b3843c458d06c1d57b in term 1.
I20260811 18:55:01.162148 29578 leader_election.cc:302] T cff19b6da213433d8ab4bd7b044848ab P b51b2678046242b3843c458d06c1d57b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 437f613592ec495090db03fb118a62b5, b51b2678046242b3843c458d06c1d57b; no voters:
I20260811 18:55:01.162324 30348 raft_consensus.cc:2781] T cff19b6da213433d8ab4bd7b044848ab P b51b2678046242b3843c458d06c1d57b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:55:01.162544 30348 raft_consensus.cc:686] T cff19b6da213433d8ab4bd7b044848ab P b51b2678046242b3843c458d06c1d57b [term 1 LEADER]: Becoming Leader. State: Replica: b51b2678046242b3843c458d06c1d57b, State: Running, Role: LEADER
I20260811 18:55:01.162853 30348 consensus_queue.cc:237] T cff19b6da213433d8ab4bd7b044848ab P b51b2678046242b3843c458d06c1d57b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b8de0e94c1c411a8c407e92312bd1a8" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 36897 } } peers { permanent_uuid: "b51b2678046242b3843c458d06c1d57b" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 37561 } } peers { permanent_uuid: "437f613592ec495090db03fb118a62b5" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 42017 } }
I20260811 18:55:01.166214 29438 catalog_manager.cc:5495] T cff19b6da213433d8ab4bd7b044848ab P b51b2678046242b3843c458d06c1d57b reported cstate change: term changed from 0 to 1, leader changed from <none> to b51b2678046242b3843c458d06c1d57b (127.27.180.129). New cstate: current_term: 1 leader_uuid: "b51b2678046242b3843c458d06c1d57b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b8de0e94c1c411a8c407e92312bd1a8" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 36897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b51b2678046242b3843c458d06c1d57b" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 37561 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "437f613592ec495090db03fb118a62b5" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 42017 } health_report { overall_health: UNKNOWN } } }
W20260811 18:55:01.171664 30313 tablet.cc:2369] T cff19b6da213433d8ab4bd7b044848ab P 0b8de0e94c1c411a8c407e92312bd1a8: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 11 18:55:01 dist-test-slave-2j7r krb5kdc[29372](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474500, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.180.129@KRBTEST.COM
I20260811 18:55:01.210747 30335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:01.201541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43858 (local address 127.27.180.129:37561)
0811 18:55:01.201705 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:01.201710 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:01.201910 (+ 200us) server_negotiation.cc:407] Connection header received
0811 18:55:01.201990 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:01.201992 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:01.202040 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:01.202113 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:01.202773 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:01.203348 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:01.204201 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:01.204443 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:01.204517 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:01.207178 (+ 2661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:01.207192 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:01.207207 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:01.207234 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:01.208874 (+ 1640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:01.209411 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:01.209417 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:01.209419 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:01.209512 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:01.209728 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:01.209730 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:01.209731 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:01.209989 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:01.210149 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:01.210424 (+ 275us) server_negotiation.cc:299] Negotiation successful
0811 18:55:01.210564 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
W20260811 18:55:01.214056 30053 tablet.cc:2369] T cff19b6da213433d8ab4bd7b044848ab P 437f613592ec495090db03fb118a62b5: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 11 18:55:01 dist-test-slave-2j7r krb5kdc[29372](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474501, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 18:55:01 dist-test-slave-2j7r krb5kdc[29372](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474501, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.27.180.129@KRBTEST.COM
I20260811 18:55:01.239964 30335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:01.231028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43866 (local address 127.27.180.129:37561)
0811 18:55:01.231220 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:01.231226 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:01.231360 (+ 134us) server_negotiation.cc:407] Connection header received
0811 18:55:01.231528 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:01.231531 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:01.231580 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:01.231661 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:01.232427 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:01.232985 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:01.233877 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:01.234125 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:01.234242 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:01.236840 (+ 2598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:01.236844 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:01.236847 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:01.236877 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:01.238381 (+ 1504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:01.238791 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:01.238794 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:01.238796 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:01.238884 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:01.239128 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:01.239132 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:01.239133 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:01.239329 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:01.239478 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:01.239677 (+ 199us) server_negotiation.cc:299] Negotiation successful
0811 18:55:01.239801 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
Aug 11 18:55:01 dist-test-slave-2j7r krb5kdc[29372](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474501, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 18:55:01 dist-test-slave-2j7r krb5kdc[29372](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474501, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.180.129@KRBTEST.COM
I20260811 18:55:01.266139 30335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:01.258281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43882 (local address 127.27.180.129:37561)
0811 18:55:01.258458 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:01.258462 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:01.258533 (+ 71us) server_negotiation.cc:407] Connection header received
0811 18:55:01.258647 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:01.258650 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:01.258694 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:01.258780 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:01.259469 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:01.259969 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:01.260719 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:01.261009 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:01.261138 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:01.263326 (+ 2188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:01.263329 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:01.263331 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:01.263356 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:01.264615 (+ 1259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:01.264983 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:01.264986 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:01.264987 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:01.265055 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:01.265321 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:01.265324 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:01.265326 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:01.265504 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:01.265633 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:01.265859 (+ 226us) server_negotiation.cc:299] Negotiation successful
0811 18:55:01.265971 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
W20260811 18:55:01.267876 29626 tablet_service.cc:2908] Rejecting scan request for tablet cff19b6da213433d8ab4bd7b044848ab: Uninitialized: clean time has not yet been initialized
I20260811 18:55:01.270398 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 29546
I20260811 18:55:01.301062 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 29800
I20260811 18:55:01.313509 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 30063
I20260811 18:55:01.324338 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 29392
2026-08-11T18:55:01Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2588 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:55:01 dist-test-slave-2j7r krb5kdc[30388](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:55:01 dist-test-slave-2j7r krb5kdc[30388](info): set up 2 sockets
Aug 11 18:55:01 dist-test-slave-2j7r krb5kdc[30388](info): commencing operation
krb5kdc: starting...
W20260811 18:55:02.517642 28370 mini_kdc.cc:121] Time spent starting KDC: real 1.162s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/krb5kdc/test-user.keytab.
Aug 11 18:55:02 dist-test-slave-2j7r krb5kdc[30388](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474502, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:55:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:55:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.180.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.180.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.180.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab.
Entry for principal kudu/127.27.180.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab.
I20260811 18:55:02.650410 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.190:34039
--webserver_interface=127.27.180.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab
--principal=kudu/127.27.180.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:40629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.180.190:34039
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:55:02.760280 30408 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:55:02.760529 30408 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:55:02.760567 30408 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:55:02.764460 30408 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:55:02.764520 30408 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 18:55:02.764539 30408 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:55:02.764555 30408 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:55:02.764569 30408 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:55:02.764583 30408 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:55:02.769717 30408 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:40629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.180.190:34039
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab
--principal=kudu/127.27.180.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.190:34039
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.180.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30408
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:55:02.770641 30408 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:55:02.771564 30408 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:55:02.779594 30413 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:02.779605 30414 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:55:02.779752 30408 server_base.cc:1034] running on GCE node
W20260811 18:55:02.779618 30416 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:55:02.780428 30408 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:55:02.781517 30408 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:55:02.782698 30408 hybrid_clock.cc:648] HybridClock initialized: now 1786474502782669 us; error 46 us; skew 500 ppm
Aug 11 18:55:02 dist-test-slave-2j7r krb5kdc[30388](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474502, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:55:02.789765 30408 init.cc:377] Logged in from keytab as kudu/127.27.180.190@KRBTEST.COM (short username kudu)
I20260811 18:55:02.790658 30408 webserver.cc:460] Webserver started at http://127.27.180.190:41261/ using document root <none> and password file <none>
I20260811 18:55:02.791200 30408 fs_manager.cc:359] Metadata directory not provided
I20260811 18:55:02.791267 30408 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:55:02.791460 30408 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:55:02.793511 30408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/minicluster-data/master-0/data/instance:
uuid: "ad3417ccb7ca4db993c61829c1f3bdbe"
format_stamp: "Formatted at 2026-08-11 18:55:02 on dist-test-slave-2j7r"
server_key: "8b06d1897231e9e64540f57ad4d8a363"
server_key_iv: "cfe14257dc3223fe5b7a0d5a88efc9b6"
server_key_version: "encryptionkey@0"
I20260811 18:55:02.794049 30408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/minicluster-data/master-0/wal/instance:
uuid: "ad3417ccb7ca4db993c61829c1f3bdbe"
format_stamp: "Formatted at 2026-08-11 18:55:02 on dist-test-slave-2j7r"
server_key: "8b06d1897231e9e64540f57ad4d8a363"
server_key_iv: "cfe14257dc3223fe5b7a0d5a88efc9b6"
server_key_version: "encryptionkey@0"
I20260811 18:55:02.797670 30408 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:55:02.799925 30424 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:02.801132 30408 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 18:55:02.801258 30408 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/minicluster-data/master-0/wal
uuid: "ad3417ccb7ca4db993c61829c1f3bdbe"
format_stamp: "Formatted at 2026-08-11 18:55:02 on dist-test-slave-2j7r"
server_key: "8b06d1897231e9e64540f57ad4d8a363"
server_key_iv: "cfe14257dc3223fe5b7a0d5a88efc9b6"
server_key_version: "encryptionkey@0"
I20260811 18:55:02.801352 30408 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:55:02.819514 30408 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:55:02.822489 30408 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:55:02.822657 30408 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:55:02.830582 30408 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.190:34039
I20260811 18:55:02.830600 30557 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.190:34039 every 8 connection(s)
I20260811 18:55:02.831506 30408 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
I20260811 18:55:02.834235 30558 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:55:02.836808 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 30408
I20260811 18:55:02.836936 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/minicluster-data/master-0/wal/instance
I20260811 18:55:02.837301 28370 external_mini_cluster.cc:1428] Setting key a12cfba3581bc3cc6f6adf50fef28949
I20260811 18:55:02.840309 30558 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad3417ccb7ca4db993c61829c1f3bdbe: Bootstrap starting.
I20260811 18:55:02.842898 30558 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ad3417ccb7ca4db993c61829c1f3bdbe: Neither blocks nor log segments found. Creating new log.
I20260811 18:55:02.843750 30558 log.cc:824] T 00000000000000000000000000000000 P ad3417ccb7ca4db993c61829c1f3bdbe: Log is configured to *not* fsync() on all Append() calls
I20260811 18:55:02.845660 30558 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad3417ccb7ca4db993c61829c1f3bdbe: No bootstrap required, opened a new log
Aug 11 18:55:02 dist-test-slave-2j7r krb5kdc[30388](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474502, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM
I20260811 18:55:02.850242 30558 raft_consensus.cc:348] T 00000000000000000000000000000000 P ad3417ccb7ca4db993c61829c1f3bdbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad3417ccb7ca4db993c61829c1f3bdbe" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 34039 } }
I20260811 18:55:02.850539 30558 raft_consensus.cc:374] T 00000000000000000000000000000000 P ad3417ccb7ca4db993c61829c1f3bdbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:55:02.850647 30558 raft_consensus.cc:729] T 00000000000000000000000000000000 P ad3417ccb7ca4db993c61829c1f3bdbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad3417ccb7ca4db993c61829c1f3bdbe, State: Initialized, Role: FOLLOWER
I20260811 18:55:02.851183 30558 consensus_queue.cc:260] T 00000000000000000000000000000000 P ad3417ccb7ca4db993c61829c1f3bdbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad3417ccb7ca4db993c61829c1f3bdbe" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 34039 } }
I20260811 18:55:02.851343 30558 raft_consensus.cc:388] T 00000000000000000000000000000000 P ad3417ccb7ca4db993c61829c1f3bdbe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:55:02.851418 30558 raft_consensus.cc:482] T 00000000000000000000000000000000 P ad3417ccb7ca4db993c61829c1f3bdbe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:55:02.851521 30558 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ad3417ccb7ca4db993c61829c1f3bdbe [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:55:02.851821 30563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:02.838654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50768 (local address 127.27.180.190:34039)
0811 18:55:02.839048 (+ 394us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:02.839063 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:02.839096 (+ 33us) server_negotiation.cc:407] Connection header received
0811 18:55:02.839733 (+ 637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:02.839749 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:02.840066 (+ 317us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:02.840334 (+ 268us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:02.841366 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:02.842381 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:02.843581 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:02.844066 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:02.844217 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:02.846918 (+ 2701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:02.846925 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:02.846941 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:02.846973 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:02.848890 (+ 1917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:02.849344 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:02.849351 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:02.849357 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:02.849522 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:02.849776 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:02.849779 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:02.849781 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:02.850285 (+ 504us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:02.850489 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:02.850770 (+ 281us) server_negotiation.cc:299] Negotiation successful
0811 18:55:02.851053 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":263,"thread_start_us":102,"threads_started":1}
I20260811 18:55:02.852456 30558 raft_consensus.cc:504] T 00000000000000000000000000000000 P ad3417ccb7ca4db993c61829c1f3bdbe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad3417ccb7ca4db993c61829c1f3bdbe" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 34039 } }
I20260811 18:55:02.852830 30558 leader_election.cc:302] T 00000000000000000000000000000000 P ad3417ccb7ca4db993c61829c1f3bdbe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ad3417ccb7ca4db993c61829c1f3bdbe; no voters:
I20260811 18:55:02.853226 30558 leader_election.cc:288] T 00000000000000000000000000000000 P ad3417ccb7ca4db993c61829c1f3bdbe [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:55:02.853334 30566 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ad3417ccb7ca4db993c61829c1f3bdbe [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:55:02.853554 30566 raft_consensus.cc:686] T 00000000000000000000000000000000 P ad3417ccb7ca4db993c61829c1f3bdbe [term 1 LEADER]: Becoming Leader. State: Replica: ad3417ccb7ca4db993c61829c1f3bdbe, State: Running, Role: LEADER
I20260811 18:55:02.853896 30566 consensus_queue.cc:237] T 00000000000000000000000000000000 P ad3417ccb7ca4db993c61829c1f3bdbe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad3417ccb7ca4db993c61829c1f3bdbe" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 34039 } }
I20260811 18:55:02.854444 30558 sys_catalog.cc:564] T 00000000000000000000000000000000 P ad3417ccb7ca4db993c61829c1f3bdbe [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:55:02.855602 30567 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad3417ccb7ca4db993c61829c1f3bdbe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ad3417ccb7ca4db993c61829c1f3bdbe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad3417ccb7ca4db993c61829c1f3bdbe" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 34039 } } }
I20260811 18:55:02.855741 30567 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad3417ccb7ca4db993c61829c1f3bdbe [sys.catalog]: This master's current role is: LEADER
I20260811 18:55:02.856068 30569 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad3417ccb7ca4db993c61829c1f3bdbe [sys.catalog]: SysCatalogTable state changed. Reason: New leader ad3417ccb7ca4db993c61829c1f3bdbe. Latest consensus state: current_term: 1 leader_uuid: "ad3417ccb7ca4db993c61829c1f3bdbe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad3417ccb7ca4db993c61829c1f3bdbe" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 34039 } } }
I20260811 18:55:02.856135 30578 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:55:02.856798 30569 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad3417ccb7ca4db993c61829c1f3bdbe [sys.catalog]: This master's current role is: LEADER
I20260811 18:55:02.858986 30578 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:55:02.866492 30578 catalog_manager.cc:1347] Generated new cluster ID: b0e72a43debf4e1c825cb69faa7814f4
I20260811 18:55:02.866566 30578 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:55:02.873781 30578 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:55:02.874656 30578 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:55:02.882361 30578 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ad3417ccb7ca4db993c61829c1f3bdbe: Generated new TSK 0
I20260811 18:55:02.882993 30578 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.180.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.180.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.180.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.129.keytab.
Entry for principal kudu/127.27.180.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.129.keytab.
I20260811 18:55:02.912871 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.180.129:0
--local_ip_for_outbound_sockets=127.27.180.129
--webserver_interface=127.27.180.129
--webserver_port=0
--tserver_master_addrs=127.27.180.190:34039
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.129.keytab
--principal=kudu/127.27.180.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:40629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:55:03.024300 30597 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:55:03.024545 30597 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:55:03.024605 30597 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:55:03.028558 30597 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:55:03.028609 30597 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:55:03.028661 30597 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 18:55:03.028712 30597 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.180.129
I20260811 18:55:03.033757 30597 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:40629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.129.keytab
--principal=kudu/127.27.180.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.180.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.180.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.27.180.190:34039
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30597
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.180.129
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:55:03.034720 30597 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:55:03.035703 30597 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:55:03.044607 30608 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:03.044752 30613 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:03.044665 30607 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:55:03.044904 30597 server_base.cc:1034] running on GCE node
I20260811 18:55:03.045336 30597 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:55:03.045898 30597 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:55:03.047057 30597 hybrid_clock.cc:648] HybridClock initialized: now 1786474503047035 us; error 35 us; skew 500 ppm
Aug 11 18:55:03 dist-test-slave-2j7r krb5kdc[30388](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474503, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:55:03.049616 30597 init.cc:377] Logged in from keytab as kudu/127.27.180.129@KRBTEST.COM (short username kudu)
I20260811 18:55:03.050381 30597 webserver.cc:460] Webserver started at http://127.27.180.129:35849/ using document root <none> and password file <none>
I20260811 18:55:03.050864 30597 fs_manager.cc:359] Metadata directory not provided
I20260811 18:55:03.050923 30597 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:55:03.051085 30597 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:55:03.053021 30597 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/minicluster-data/ts-0/data/instance:
uuid: "76b321a2e8c3434a8a225f70d4b24567"
format_stamp: "Formatted at 2026-08-11 18:55:03 on dist-test-slave-2j7r"
server_key: "ffdc9ecb8a5ce846020c1135890401c4"
server_key_iv: "101bf8f73c9260bd7e8c8dba51ae5f21"
server_key_version: "encryptionkey@0"
I20260811 18:55:03.053594 30597 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/minicluster-data/ts-0/wal/instance:
uuid: "76b321a2e8c3434a8a225f70d4b24567"
format_stamp: "Formatted at 2026-08-11 18:55:03 on dist-test-slave-2j7r"
server_key: "ffdc9ecb8a5ce846020c1135890401c4"
server_key_iv: "101bf8f73c9260bd7e8c8dba51ae5f21"
server_key_version: "encryptionkey@0"
I20260811 18:55:03.057390 30597 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 18:55:03.059743 30626 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:03.060895 30597 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 18:55:03.060992 30597 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/minicluster-data/ts-0/wal
uuid: "76b321a2e8c3434a8a225f70d4b24567"
format_stamp: "Formatted at 2026-08-11 18:55:03 on dist-test-slave-2j7r"
server_key: "ffdc9ecb8a5ce846020c1135890401c4"
server_key_iv: "101bf8f73c9260bd7e8c8dba51ae5f21"
server_key_version: "encryptionkey@0"
I20260811 18:55:03.061074 30597 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:55:03.089289 30597 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:55:03.092365 30597 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:55:03.092517 30597 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:55:03.094012 30597 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:55:03.094062 30597 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:03.094105 30597 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:55:03.094133 30597 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 18:55:03 dist-test-slave-2j7r krb5kdc[30388](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474503, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.129@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM
I20260811 18:55:03.104172 30597 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.129:38053
I20260811 18:55:03.104627 30835 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.129:38053 every 8 connection(s)
I20260811 18:55:03.105182 30597 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/minicluster-data/ts-0/data/info.pb
I20260811 18:55:03.107491 30563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:03.094775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.129:36015 (local address 127.27.180.190:34039)
0811 18:55:03.094902 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:03.094907 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:03.095821 (+ 914us) server_negotiation.cc:407] Connection header received
0811 18:55:03.096902 (+ 1081us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:03.096905 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:03.096965 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:03.097024 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:03.098737 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:03.099575 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:03.100818 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:03.101042 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:03.101175 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:03.103670 (+ 2495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:03.103673 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:03.103675 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:03.103699 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:03.105383 (+ 1684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:03.105989 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:03.105993 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:03.105995 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:03.106089 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:03.106430 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:03.106433 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:03.106435 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:03.106616 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:03.106696 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:03.107194 (+ 498us) server_negotiation.cc:299] Negotiation successful
0811 18:55:03.107326 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 18:55:03.108197 30646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:03.095176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:34039 (local address 127.27.180.129:36015)
0811 18:55:03.095634 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:03.095684 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:03.096644 (+ 960us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:03.097457 (+ 813us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:03.097466 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:03.097867 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:03.098539 (+ 672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:03.098559 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:03.099723 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:03.099727 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:03.100689 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:03.100699 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:03.101228 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:03.101233 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:03.101407 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:03.101942 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:03.101958 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:03.103491 (+ 1533us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:55:03.105491 (+ 2000us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:03.105518 (+ 27us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:03.105532 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:03.105872 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:03.106190 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:03.106193 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:03.106195 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:03.106333 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:03.106731 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:55:03.106737 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:55:03.106968 (+ 231us) client_negotiation.cc:769] Sending connection context
0811 18:55:03.107126 (+ 158us) client_negotiation.cc:240] Negotiation successful
0811 18:55:03.107425 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":108,"threads_started":1}
I20260811 18:55:03.109349 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 30597
I20260811 18:55:03.109474 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/minicluster-data/ts-0/wal/instance
I20260811 18:55:03.109539 30836 heartbeater.cc:347] Connected to a master server at 127.27.180.190:34039
I20260811 18:55:03.109690 30836 heartbeater.cc:464] Registering TS with master...
I20260811 18:55:03.109843 28370 external_mini_cluster.cc:1428] Setting key d5f6b4e1a076c26c28263b1fa32e2bee
I20260811 18:55:03.110217 30836 heartbeater.cc:511] Master 127.27.180.190:34039 requested a full tablet report, sending...
I20260811 18:55:03.112152 30474 ts_manager.cc:194] Registered new tserver with Master: 76b321a2e8c3434a8a225f70d4b24567 (127.27.180.129:38053)
I20260811 18:55:03.113579 30474 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.180.129@KRBTEST.COM'} at 127.27.180.129:36015
I20260811 18:55:03.119629 30844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:03.111911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:34039 (local address 127.27.180.129:56993)
0811 18:55:03.112251 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:03.112267 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:03.112404 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:03.112651 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:03.112654 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:03.112856 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:03.112966 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:03.112972 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:03.113914 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:03.113917 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:03.114749 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:03.114760 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:03.115336 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:03.115340 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:03.115444 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:03.115958 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:03.115973 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:03.116344 (+ 371us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:55:03.118219 (+ 1875us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:03.118222 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:03.118224 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:03.118434 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:03.118720 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:03.118722 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:03.118723 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:03.118788 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:03.119214 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:55:03.119215 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:55:03.119287 (+ 72us) client_negotiation.cc:769] Sending connection context
0811 18:55:03.119382 (+ 95us) client_negotiation.cc:240] Negotiation successful
0811 18:55:03.119485 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"mutex_wait_us":3,"thread_start_us":106,"threads_started":1}
I20260811 18:55:03.119766 30563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:03.111981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.129:56993 (local address 127.27.180.190:34039)
0811 18:55:03.112107 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:03.112112 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:03.112336 (+ 224us) server_negotiation.cc:407] Connection header received
0811 18:55:03.112498 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:03.112502 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:03.112548 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:03.112621 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:03.113108 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:03.113760 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:03.114885 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:03.115194 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:03.115307 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:03.116482 (+ 1175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:03.116487 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:03.116490 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:03.116518 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:03.118098 (+ 1580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:03.118530 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:03.118535 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:03.118537 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:03.118633 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:03.118871 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:03.118874 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:03.118876 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:03.119102 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:03.119238 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:03.119467 (+ 229us) server_negotiation.cc:299] Negotiation successful
0811 18:55:03.119594 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
WARNING: no policy specified for kudu/127.27.180.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.180.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.180.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.130.keytab.
Entry for principal kudu/127.27.180.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.130.keytab.
I20260811 18:55:03.138062 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.180.130:0
--local_ip_for_outbound_sockets=127.27.180.130
--webserver_interface=127.27.180.130
--webserver_port=0
--tserver_master_addrs=127.27.180.190:34039
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.130.keytab
--principal=kudu/127.27.180.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:40629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:55:03.246179 30849 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:55:03.246400 30849 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:55:03.246441 30849 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:55:03.250265 30849 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:55:03.250309 30849 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:55:03.250344 30849 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 18:55:03.250380 30849 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.180.130
I20260811 18:55:03.255442 30849 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:40629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.130.keytab
--principal=kudu/127.27.180.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.180.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.180.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.27.180.190:34039
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30849
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.180.130
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:55:03.256382 30849 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:55:03.257293 30849 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:55:03.266505 30860 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:03.266526 30871 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:03.266543 30864 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:55:03.266873 30849 server_base.cc:1034] running on GCE node
I20260811 18:55:03.267305 30849 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:55:03.267884 30849 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:55:03.269074 30849 hybrid_clock.cc:648] HybridClock initialized: now 1786474503269060 us; error 23 us; skew 500 ppm
Aug 11 18:55:03 dist-test-slave-2j7r krb5kdc[30388](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474503, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:55:03.271737 30849 init.cc:377] Logged in from keytab as kudu/127.27.180.130@KRBTEST.COM (short username kudu)
I20260811 18:55:03.272528 30849 webserver.cc:460] Webserver started at http://127.27.180.130:43961/ using document root <none> and password file <none>
I20260811 18:55:03.273016 30849 fs_manager.cc:359] Metadata directory not provided
I20260811 18:55:03.273072 30849 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:55:03.273308 30849 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:55:03.275261 30849 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/minicluster-data/ts-1/data/instance:
uuid: "04474360ddcb4b2a8686a22455345733"
format_stamp: "Formatted at 2026-08-11 18:55:03 on dist-test-slave-2j7r"
server_key: "4a21b7daf496e38a544b265644d8b7c3"
server_key_iv: "bd776347c0ec45a2acbf7b07a3e2d488"
server_key_version: "encryptionkey@0"
I20260811 18:55:03.275755 30849 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/minicluster-data/ts-1/wal/instance:
uuid: "04474360ddcb4b2a8686a22455345733"
format_stamp: "Formatted at 2026-08-11 18:55:03 on dist-test-slave-2j7r"
server_key: "4a21b7daf496e38a544b265644d8b7c3"
server_key_iv: "bd776347c0ec45a2acbf7b07a3e2d488"
server_key_version: "encryptionkey@0"
I20260811 18:55:03.279222 30849 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 18:55:03.281499 30886 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:03.282608 30849 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 18:55:03.282718 30849 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/minicluster-data/ts-1/wal
uuid: "04474360ddcb4b2a8686a22455345733"
format_stamp: "Formatted at 2026-08-11 18:55:03 on dist-test-slave-2j7r"
server_key: "4a21b7daf496e38a544b265644d8b7c3"
server_key_iv: "bd776347c0ec45a2acbf7b07a3e2d488"
server_key_version: "encryptionkey@0"
I20260811 18:55:03.282825 30849 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:55:03.328377 30849 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:55:03.331920 30849 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:55:03.332105 30849 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:55:03.333648 30849 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:55:03.333698 30849 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:03.333741 30849 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:55:03.333766 30849 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 18:55:03 dist-test-slave-2j7r krb5kdc[30388](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474503, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.130@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM
I20260811 18:55:03.344281 30849 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.130:43235
I20260811 18:55:03.344398 31106 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.130:43235 every 8 connection(s)
I20260811 18:55:03.345366 30849 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/minicluster-data/ts-1/data/info.pb
I20260811 18:55:03.346791 30563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:03.334443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.130:53947 (local address 127.27.180.190:34039)
0811 18:55:03.334563 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:03.334567 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:03.335623 (+ 1056us) server_negotiation.cc:407] Connection header received
0811 18:55:03.336602 (+ 979us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:03.336606 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:03.336666 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:03.336722 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:03.338219 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:03.338866 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:03.340071 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:03.340325 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:03.340424 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:03.343105 (+ 2681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:03.343109 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:03.343111 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:03.343140 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:03.344640 (+ 1500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:03.345211 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:03.345214 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:03.345215 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:03.345319 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:03.345677 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:03.345679 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:03.345680 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:03.345901 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:03.346013 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:03.346561 (+ 548us) server_negotiation.cc:299] Negotiation successful
0811 18:55:03.346666 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 18:55:03.347527 30904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:03.334959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:34039 (local address 127.27.180.130:53947)
0811 18:55:03.335468 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:03.335511 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:03.336351 (+ 840us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:03.337124 (+ 773us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:03.337132 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:03.337537 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:03.338062 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:03.338078 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:03.338990 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:03.338993 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:03.339937 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:03.339947 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:03.340475 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:03.340479 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:03.340638 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:03.341175 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:03.341191 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:03.342923 (+ 1732us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:55:03.344751 (+ 1828us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:03.344757 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:03.344770 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:03.345119 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:03.345416 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:03.345419 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:03.345421 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:03.345582 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:03.346067 (+ 485us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:55:03.346073 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:55:03.346338 (+ 265us) client_negotiation.cc:769] Sending connection context
0811 18:55:03.346490 (+ 152us) client_negotiation.cc:240] Negotiation successful
0811 18:55:03.346723 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":136,"threads_started":1}
I20260811 18:55:03.348634 31107 heartbeater.cc:347] Connected to a master server at 127.27.180.190:34039
I20260811 18:55:03.348775 31107 heartbeater.cc:464] Registering TS with master...
I20260811 18:55:03.349316 31107 heartbeater.cc:511] Master 127.27.180.190:34039 requested a full tablet report, sending...
I20260811 18:55:03.350296 30474 ts_manager.cc:194] Registered new tserver with Master: 04474360ddcb4b2a8686a22455345733 (127.27.180.130:43235)
I20260811 18:55:03.350906 30474 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.180.130@KRBTEST.COM'} at 127.27.180.130:53947
I20260811 18:55:03.354614 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 30849
I20260811 18:55:03.354722 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/minicluster-data/ts-1/wal/instance
I20260811 18:55:03.354996 28370 external_mini_cluster.cc:1428] Setting key 600b9df0debcc9a07e610c7c6ef29de9
I20260811 18:55:03.357714 30563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:03.350742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.130:59163 (local address 127.27.180.190:34039)
0811 18:55:03.350845 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:03.350849 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:03.351031 (+ 182us) server_negotiation.cc:407] Connection header received
0811 18:55:03.351070 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:03.351073 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:03.351114 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:03.351189 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:03.351730 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:03.352457 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:03.353336 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:03.353541 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:03.353621 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:03.354675 (+ 1054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:03.354677 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:03.354679 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:03.354700 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:03.356040 (+ 1340us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:03.356477 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:03.356480 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:03.356482 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:03.356578 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:03.356880 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:03.356883 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:03.356885 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:03.357110 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:03.357183 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:03.357478 (+ 295us) server_negotiation.cc:299] Negotiation successful
0811 18:55:03.357575 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 18:55:03.357728 31113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:03.350656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:34039 (local address 127.27.180.130:59163)
0811 18:55:03.350904 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:03.350915 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:03.351008 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:03.351295 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:03.351298 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:03.351492 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:03.351617 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:03.351623 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:03.352606 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:03.352609 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:03.353215 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:03.353226 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:03.353678 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:03.353681 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:03.353772 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:03.354149 (+ 377us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:03.354159 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:03.354553 (+ 394us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:55:03.356168 (+ 1615us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:03.356172 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:03.356173 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:03.356365 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:03.356660 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:03.356663 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:03.356665 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:03.356754 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:03.357243 (+ 489us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:55:03.357246 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:55:03.357320 (+ 74us) client_negotiation.cc:769] Sending connection context
0811 18:55:03.357402 (+ 82us) client_negotiation.cc:240] Negotiation successful
0811 18:55:03.357536 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"mutex_wait_us":5,"thread_start_us":83,"threads_started":1}
WARNING: no policy specified for kudu/127.27.180.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.180.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.180.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.131.keytab.
Entry for principal kudu/127.27.180.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.131.keytab.
I20260811 18:55:03.381927 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.180.131:0
--local_ip_for_outbound_sockets=127.27.180.131
--webserver_interface=127.27.180.131
--webserver_port=0
--tserver_master_addrs=127.27.180.190:34039
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.131.keytab
--principal=kudu/127.27.180.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:40629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:55:03.495616 31121 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:55:03.495863 31121 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:55:03.495924 31121 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:55:03.499873 31121 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:55:03.499929 31121 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:55:03.499980 31121 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 18:55:03.500033 31121 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.180.131
I20260811 18:55:03.505568 31121 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:40629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.131.keytab
--principal=kudu/127.27.180.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.180.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.180.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.27.180.190:34039
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31121
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.180.131
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:55:03.506541 31121 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:55:03.507535 31121 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:55:03.516713 31132 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:03.516779 31130 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:03.516747 31139 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:55:03.516994 31121 server_base.cc:1034] running on GCE node
I20260811 18:55:03.517416 31121 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:55:03.518009 31121 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:55:03.519197 31121 hybrid_clock.cc:648] HybridClock initialized: now 1786474503519178 us; error 25 us; skew 500 ppm
Aug 11 18:55:03 dist-test-slave-2j7r krb5kdc[30388](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474503, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:55:03.521885 31121 init.cc:377] Logged in from keytab as kudu/127.27.180.131@KRBTEST.COM (short username kudu)
I20260811 18:55:03.522678 31121 webserver.cc:460] Webserver started at http://127.27.180.131:44925/ using document root <none> and password file <none>
I20260811 18:55:03.523203 31121 fs_manager.cc:359] Metadata directory not provided
I20260811 18:55:03.523268 31121 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:55:03.523460 31121 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:55:03.525471 31121 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/minicluster-data/ts-2/data/instance:
uuid: "76aa1f06e7874cde9accdecc25c56921"
format_stamp: "Formatted at 2026-08-11 18:55:03 on dist-test-slave-2j7r"
server_key: "b78dda615b6e9df7e73ce60b295e2abe"
server_key_iv: "deda39ed1a9dab437dd25745a10bf98b"
server_key_version: "encryptionkey@0"
I20260811 18:55:03.526021 31121 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/minicluster-data/ts-2/wal/instance:
uuid: "76aa1f06e7874cde9accdecc25c56921"
format_stamp: "Formatted at 2026-08-11 18:55:03 on dist-test-slave-2j7r"
server_key: "b78dda615b6e9df7e73ce60b295e2abe"
server_key_iv: "deda39ed1a9dab437dd25745a10bf98b"
server_key_version: "encryptionkey@0"
I20260811 18:55:03.529613 31121 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 18:55:03.531862 31152 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:03.533015 31121 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 18:55:03.533141 31121 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/minicluster-data/ts-2/wal
uuid: "76aa1f06e7874cde9accdecc25c56921"
format_stamp: "Formatted at 2026-08-11 18:55:03 on dist-test-slave-2j7r"
server_key: "b78dda615b6e9df7e73ce60b295e2abe"
server_key_iv: "deda39ed1a9dab437dd25745a10bf98b"
server_key_version: "encryptionkey@0"
I20260811 18:55:03.533241 31121 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:55:03.547698 31121 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:55:03.550777 31121 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:55:03.550956 31121 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:55:03.552459 31121 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:55:03.552539 31121 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:03.552605 31121 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:55:03.552635 31121 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 18:55:03 dist-test-slave-2j7r krb5kdc[30388](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474503, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.131@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM
I20260811 18:55:03.563766 31121 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.131:46825
I20260811 18:55:03.563827 31351 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.131:46825 every 8 connection(s)
I20260811 18:55:03.564867 31121 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/minicluster-data/ts-2/data/info.pb
I20260811 18:55:03.566377 30563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:03.553367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.131:51853 (local address 127.27.180.190:34039)
0811 18:55:03.553492 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:03.553495 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:03.554592 (+ 1097us) server_negotiation.cc:407] Connection header received
0811 18:55:03.555552 (+ 960us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:03.555556 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:03.555618 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:03.555715 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:03.557136 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:03.557927 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:03.559245 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:03.559560 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:03.559655 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:03.562544 (+ 2889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:03.562548 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:03.562551 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:03.562581 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:03.564181 (+ 1600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:03.564680 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:03.564684 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:03.564686 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:03.564793 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:03.565144 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:03.565147 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:03.565149 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:03.565382 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:03.565502 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:03.566146 (+ 644us) server_negotiation.cc:299] Negotiation successful
0811 18:55:03.566258 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 18:55:03.567073 31168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:03.553919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:34039 (local address 127.27.180.131:51853)
0811 18:55:03.554423 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:03.554464 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:03.555315 (+ 851us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:03.555920 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:03.555928 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:03.556408 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:03.556984 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:03.557000 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:03.558076 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:03.558079 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:03.559090 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:03.559101 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:03.559747 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:03.559752 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:03.559943 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:03.560492 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:03.560510 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:03.562373 (+ 1863us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:55:03.564305 (+ 1932us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:03.564311 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:03.564325 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:03.564576 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:03.564875 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:03.564877 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:03.564878 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:03.565039 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:03.565579 (+ 540us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:55:03.565585 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:55:03.565884 (+ 299us) client_negotiation.cc:769] Sending connection context
0811 18:55:03.566065 (+ 181us) client_negotiation.cc:240] Negotiation successful
0811 18:55:03.566303 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":145,"threads_started":1}
I20260811 18:55:03.567793 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 31121
I20260811 18:55:03.567919 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/minicluster-data/ts-2/wal/instance
I20260811 18:55:03.568202 28370 external_mini_cluster.cc:1428] Setting key 9da7f04b7144b7ddcd16cc2103740094
I20260811 18:55:03.568434 31352 heartbeater.cc:347] Connected to a master server at 127.27.180.190:34039
I20260811 18:55:03.568588 31352 heartbeater.cc:464] Registering TS with master...
I20260811 18:55:03.569206 31352 heartbeater.cc:511] Master 127.27.180.190:34039 requested a full tablet report, sending...
I20260811 18:55:03.570322 30477 ts_manager.cc:194] Registered new tserver with Master: 76aa1f06e7874cde9accdecc25c56921 (127.27.180.131:46825)
I20260811 18:55:03.570950 30477 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.180.131@KRBTEST.COM'} at 127.27.180.131:51853
I20260811 18:55:03.576918 31359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:03.570526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:34039 (local address 127.27.180.131:35479)
0811 18:55:03.570841 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:03.570852 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:03.570942 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:03.571139 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:03.571142 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:03.571332 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:03.571428 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:03.571434 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:03.572172 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:03.572176 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:03.572723 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:03.572732 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:03.573197 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:03.573201 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:03.573312 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:03.573733 (+ 421us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:03.573745 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:03.574036 (+ 291us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:55:03.575516 (+ 1480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:03.575519 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:03.575521 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:03.575714 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:03.575983 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:03.575986 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:03.575988 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:03.576067 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:03.576422 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:55:03.576425 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:55:03.576514 (+ 89us) client_negotiation.cc:769] Sending connection context
0811 18:55:03.576602 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 18:55:03.576738 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":90,"threads_started":1}
I20260811 18:55:03.576917 30563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:03.570609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.131:35479 (local address 127.27.180.190:34039)
0811 18:55:03.570741 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:03.570747 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:03.570917 (+ 170us) server_negotiation.cc:407] Connection header received
0811 18:55:03.571011 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:03.571014 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:03.571057 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:03.571122 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:03.571538 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:03.572048 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:03.572832 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:03.573052 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:03.573192 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:03.574174 (+ 982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:03.574178 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:03.574180 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:03.574206 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:03.575416 (+ 1210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:03.575808 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:03.575811 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:03.575813 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:03.575890 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:03.576150 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:03.576154 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:03.576156 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:03.576326 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:03.576419 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:03.576674 (+ 255us) server_negotiation.cc:299] Negotiation successful
0811 18:55:03.576790 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 18:55:03.583078 28370 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 18:55:03 dist-test-slave-2j7r krb5kdc[30388](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474503, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 18:55:03 dist-test-slave-2j7r krb5kdc[30388](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474503, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM
I20260811 18:55:03.608626 30563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:03.601022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50778 (local address 127.27.180.190:34039)
0811 18:55:03.601192 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:03.601196 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:03.601212 (+ 16us) server_negotiation.cc:407] Connection header received
0811 18:55:03.601255 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:03.601258 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:03.601299 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:03.601365 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:03.602066 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:03.602580 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:03.603411 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:03.603633 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:03.603718 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:03.606049 (+ 2331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:03.606054 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:03.606056 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:03.606087 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:03.607342 (+ 1255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:03.607690 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:03.607692 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:03.607693 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:03.607774 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:03.607961 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:03.607963 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:03.607964 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:03.608119 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:03.608211 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:03.608377 (+ 166us) server_negotiation.cc:299] Negotiation successful
0811 18:55:03.608466 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 18:55:03.611625 30477 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50778:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:55:03.614619 30477 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:55:03.629267 31379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:03.623280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.130:43235 (local address 127.0.0.1:37596)
0811 18:55:03.623635 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:03.623688 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:03.623829 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:03.624789 (+ 960us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:03.624812 (+ 23us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:03.624834 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:55:03.624957 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:03.624964 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:03.626222 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:03.626226 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:03.627693 (+ 1467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:03.627707 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:03.628688 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:03.628692 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:03.628815 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:03.628855 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 18:55:03.628961 (+ 106us) client_negotiation.cc:240] Negotiation successful
0811 18:55:03.629079 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"mutex_wait_us":50,"thread_start_us":86,"threads_started":1}
I20260811 18:55:03.629436 31384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:03.624252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37806 (local address 127.27.180.129:38053)
0811 18:55:03.624982 (+ 730us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:03.624988 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:03.625011 (+ 23us) server_negotiation.cc:407] Connection header received
0811 18:55:03.625080 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:03.625143 (+ 63us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:03.625320 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:03.625449 (+ 129us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:55:03.625822 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:03.626751 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:03.628017 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:03.628801 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:03.628933 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:03.629067 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:03.629162 (+ 95us) server_negotiation.cc:299] Negotiation successful
0811 18:55:03.629279 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":630,"thread_start_us":79,"threads_started":1}
I20260811 18:55:03.629590 31380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:03.623447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37596 (local address 127.27.180.130:43235)
0811 18:55:03.624270 (+ 823us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:03.624283 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:03.624321 (+ 38us) server_negotiation.cc:407] Connection header received
0811 18:55:03.624401 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:03.624406 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:03.624622 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:03.624807 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:55:03.625065 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:03.626100 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:03.627853 (+ 1753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:03.628549 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:03.628696 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:03.628848 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:03.629356 (+ 508us) server_negotiation.cc:299] Negotiation successful
0811 18:55:03.629455 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":710,"thread_start_us":96,"threads_started":1}
I20260811 18:55:03.629274 31383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:03.623799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.129:38053 (local address 127.0.0.1:37806)
0811 18:55:03.624548 (+ 749us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:03.624560 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:03.624642 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:03.625525 (+ 883us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:03.625529 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:03.625548 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:55:03.625655 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:03.625664 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:03.626885 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:03.626888 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:03.627715 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:03.627726 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:03.628930 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:03.628934 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:03.629014 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:03.629034 (+ 20us) client_negotiation.cc:769] Sending connection context
0811 18:55:03.629110 (+ 76us) client_negotiation.cc:240] Negotiation successful
0811 18:55:03.629167 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":675,"thread_start_us":102,"threads_started":1}
I20260811 18:55:03.630177 31381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:03.623997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.131:46825 (local address 127.0.0.1:35480)
0811 18:55:03.624626 (+ 629us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:03.624642 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:03.624748 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:03.625367 (+ 619us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:03.625370 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:03.625390 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:55:03.625489 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:03.625495 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:03.626710 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:03.626714 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:03.627865 (+ 1151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:03.627874 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:03.629162 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:03.629166 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:03.629269 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:03.629296 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 18:55:03.629951 (+ 655us) client_negotiation.cc:240] Negotiation successful
0811 18:55:03.630033 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":560,"mutex_wait_us":3,"thread_start_us":88,"threads_started":1}
I20260811 18:55:03.630368 31386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:03.624424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35480 (local address 127.27.180.131:46825)
0811 18:55:03.624872 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:03.624885 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:03.624910 (+ 25us) server_negotiation.cc:407] Connection header received
0811 18:55:03.624987 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:03.624993 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:03.625203 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:03.625356 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:55:03.625610 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:03.626550 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:03.627987 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:03.628801 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:03.628947 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:03.629067 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:03.630099 (+ 1032us) server_negotiation.cc:299] Negotiation successful
0811 18:55:03.630218 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":116,"threads_started":1}
I20260811 18:55:03.632623 30726 tablet_service.cc:1467] Processing CreateTablet for tablet b703eba6157b4964a79e5de799595552 (DEFAULT_TABLE table=test-table [id=a36f1d3ff01547acbaf0b57c759f3168]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:55:03.632637 30974 tablet_service.cc:1467] Processing CreateTablet for tablet b703eba6157b4964a79e5de799595552 (DEFAULT_TABLE table=test-table [id=a36f1d3ff01547acbaf0b57c759f3168]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:55:03.632843 31234 tablet_service.cc:1467] Processing CreateTablet for tablet b703eba6157b4964a79e5de799595552 (DEFAULT_TABLE table=test-table [id=a36f1d3ff01547acbaf0b57c759f3168]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:55:03.633752 31234 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b703eba6157b4964a79e5de799595552. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:55:03.633790 30974 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b703eba6157b4964a79e5de799595552. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:55:03.633790 30726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b703eba6157b4964a79e5de799595552. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:55:03.640728 31387 tablet_bootstrap.cc:492] T b703eba6157b4964a79e5de799595552 P 76aa1f06e7874cde9accdecc25c56921: Bootstrap starting.
I20260811 18:55:03.641395 31390 tablet_bootstrap.cc:492] T b703eba6157b4964a79e5de799595552 P 76b321a2e8c3434a8a225f70d4b24567: Bootstrap starting.
I20260811 18:55:03.642161 31391 tablet_bootstrap.cc:492] T b703eba6157b4964a79e5de799595552 P 04474360ddcb4b2a8686a22455345733: Bootstrap starting.
I20260811 18:55:03.643435 31387 tablet_bootstrap.cc:654] T b703eba6157b4964a79e5de799595552 P 76aa1f06e7874cde9accdecc25c56921: Neither blocks nor log segments found. Creating new log.
I20260811 18:55:03.644199 31387 log.cc:824] T b703eba6157b4964a79e5de799595552 P 76aa1f06e7874cde9accdecc25c56921: Log is configured to *not* fsync() on all Append() calls
I20260811 18:55:03.644245 31390 tablet_bootstrap.cc:654] T b703eba6157b4964a79e5de799595552 P 76b321a2e8c3434a8a225f70d4b24567: Neither blocks nor log segments found. Creating new log.
I20260811 18:55:03.644245 31391 tablet_bootstrap.cc:654] T b703eba6157b4964a79e5de799595552 P 04474360ddcb4b2a8686a22455345733: Neither blocks nor log segments found. Creating new log.
I20260811 18:55:03.645067 31391 log.cc:824] T b703eba6157b4964a79e5de799595552 P 04474360ddcb4b2a8686a22455345733: Log is configured to *not* fsync() on all Append() calls
I20260811 18:55:03.645067 31390 log.cc:824] T b703eba6157b4964a79e5de799595552 P 76b321a2e8c3434a8a225f70d4b24567: Log is configured to *not* fsync() on all Append() calls
I20260811 18:55:03.646343 31387 tablet_bootstrap.cc:492] T b703eba6157b4964a79e5de799595552 P 76aa1f06e7874cde9accdecc25c56921: No bootstrap required, opened a new log
I20260811 18:55:03.646535 31387 ts_tablet_manager.cc:1397] T b703eba6157b4964a79e5de799595552 P 76aa1f06e7874cde9accdecc25c56921: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260811 18:55:03.646953 31390 tablet_bootstrap.cc:492] T b703eba6157b4964a79e5de799595552 P 76b321a2e8c3434a8a225f70d4b24567: No bootstrap required, opened a new log
I20260811 18:55:03.646953 31391 tablet_bootstrap.cc:492] T b703eba6157b4964a79e5de799595552 P 04474360ddcb4b2a8686a22455345733: No bootstrap required, opened a new log
I20260811 18:55:03.647136 31391 ts_tablet_manager.cc:1397] T b703eba6157b4964a79e5de799595552 P 04474360ddcb4b2a8686a22455345733: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260811 18:55:03.647136 31390 ts_tablet_manager.cc:1397] T b703eba6157b4964a79e5de799595552 P 76b321a2e8c3434a8a225f70d4b24567: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.002s
I20260811 18:55:03.649765 31387 raft_consensus.cc:348] T b703eba6157b4964a79e5de799595552 P 76aa1f06e7874cde9accdecc25c56921 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04474360ddcb4b2a8686a22455345733" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 43235 } } peers { permanent_uuid: "76aa1f06e7874cde9accdecc25c56921" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 46825 } } peers { permanent_uuid: "76b321a2e8c3434a8a225f70d4b24567" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38053 } }
I20260811 18:55:03.649986 31387 raft_consensus.cc:374] T b703eba6157b4964a79e5de799595552 P 76aa1f06e7874cde9accdecc25c56921 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:55:03.650044 31387 raft_consensus.cc:729] T b703eba6157b4964a79e5de799595552 P 76aa1f06e7874cde9accdecc25c56921 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76aa1f06e7874cde9accdecc25c56921, State: Initialized, Role: FOLLOWER
I20260811 18:55:03.650476 31387 consensus_queue.cc:260] T b703eba6157b4964a79e5de799595552 P 76aa1f06e7874cde9accdecc25c56921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04474360ddcb4b2a8686a22455345733" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 43235 } } peers { permanent_uuid: "76aa1f06e7874cde9accdecc25c56921" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 46825 } } peers { permanent_uuid: "76b321a2e8c3434a8a225f70d4b24567" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38053 } }
I20260811 18:55:03.650960 31391 raft_consensus.cc:348] T b703eba6157b4964a79e5de799595552 P 04474360ddcb4b2a8686a22455345733 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04474360ddcb4b2a8686a22455345733" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 43235 } } peers { permanent_uuid: "76aa1f06e7874cde9accdecc25c56921" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 46825 } } peers { permanent_uuid: "76b321a2e8c3434a8a225f70d4b24567" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38053 } }
I20260811 18:55:03.651204 31391 raft_consensus.cc:374] T b703eba6157b4964a79e5de799595552 P 04474360ddcb4b2a8686a22455345733 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:55:03.651225 31352 heartbeater.cc:503] Master 127.27.180.190:34039 was elected leader, sending a full tablet report...
I20260811 18:55:03.651273 31391 raft_consensus.cc:729] T b703eba6157b4964a79e5de799595552 P 04474360ddcb4b2a8686a22455345733 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04474360ddcb4b2a8686a22455345733, State: Initialized, Role: FOLLOWER
I20260811 18:55:03.651683 31391 consensus_queue.cc:260] T b703eba6157b4964a79e5de799595552 P 04474360ddcb4b2a8686a22455345733 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04474360ddcb4b2a8686a22455345733" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 43235 } } peers { permanent_uuid: "76aa1f06e7874cde9accdecc25c56921" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 46825 } } peers { permanent_uuid: "76b321a2e8c3434a8a225f70d4b24567" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38053 } }
I20260811 18:55:03.652261 31387 ts_tablet_manager.cc:1428] T b703eba6157b4964a79e5de799595552 P 76aa1f06e7874cde9accdecc25c56921: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 18:55:03.652608 31391 ts_tablet_manager.cc:1428] T b703eba6157b4964a79e5de799595552 P 04474360ddcb4b2a8686a22455345733: Time spent starting tablet: real 0.005s user 0.004s sys 0.002s
I20260811 18:55:03.652953 31107 heartbeater.cc:503] Master 127.27.180.190:34039 was elected leader, sending a full tablet report...
I20260811 18:55:03.652962 31390 raft_consensus.cc:348] T b703eba6157b4964a79e5de799595552 P 76b321a2e8c3434a8a225f70d4b24567 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04474360ddcb4b2a8686a22455345733" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 43235 } } peers { permanent_uuid: "76aa1f06e7874cde9accdecc25c56921" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 46825 } } peers { permanent_uuid: "76b321a2e8c3434a8a225f70d4b24567" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38053 } }
I20260811 18:55:03.653200 31390 raft_consensus.cc:374] T b703eba6157b4964a79e5de799595552 P 76b321a2e8c3434a8a225f70d4b24567 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:55:03.653258 31390 raft_consensus.cc:729] T b703eba6157b4964a79e5de799595552 P 76b321a2e8c3434a8a225f70d4b24567 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76b321a2e8c3434a8a225f70d4b24567, State: Initialized, Role: FOLLOWER
I20260811 18:55:03.653625 31390 consensus_queue.cc:260] T b703eba6157b4964a79e5de799595552 P 76b321a2e8c3434a8a225f70d4b24567 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04474360ddcb4b2a8686a22455345733" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 43235 } } peers { permanent_uuid: "76aa1f06e7874cde9accdecc25c56921" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 46825 } } peers { permanent_uuid: "76b321a2e8c3434a8a225f70d4b24567" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38053 } }
I20260811 18:55:03.654845 30836 heartbeater.cc:503] Master 127.27.180.190:34039 was elected leader, sending a full tablet report...
I20260811 18:55:03.655107 31390 ts_tablet_manager.cc:1428] T b703eba6157b4964a79e5de799595552 P 76b321a2e8c3434a8a225f70d4b24567: Time spent starting tablet: real 0.008s user 0.004s sys 0.004s
I20260811 18:55:03.752058 31400 raft_consensus.cc:482] T b703eba6157b4964a79e5de799595552 P 04474360ddcb4b2a8686a22455345733 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:55:03.752293 31400 raft_consensus.cc:504] T b703eba6157b4964a79e5de799595552 P 04474360ddcb4b2a8686a22455345733 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04474360ddcb4b2a8686a22455345733" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 43235 } } peers { permanent_uuid: "76aa1f06e7874cde9accdecc25c56921" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 46825 } } peers { permanent_uuid: "76b321a2e8c3434a8a225f70d4b24567" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38053 } }
I20260811 18:55:03.753516 31400 leader_election.cc:288] T b703eba6157b4964a79e5de799595552 P 04474360ddcb4b2a8686a22455345733 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 76aa1f06e7874cde9accdecc25c56921 (127.27.180.131:46825), 76b321a2e8c3434a8a225f70d4b24567 (127.27.180.129:38053)
I20260811 18:55:03.758073 30904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:03.753757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.131:46825 (local address 127.27.180.130:54621)
0811 18:55:03.753943 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:03.753959 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:03.754052 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:03.754343 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:03.754346 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:03.754365 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:55:03.754484 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:03.754490 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:03.755609 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:03.755612 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:03.756954 (+ 1342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:03.756967 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:03.757728 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:03.757731 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:03.757821 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:03.757843 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 18:55:03.757902 (+ 59us) client_negotiation.cc:240] Negotiation successful
0811 18:55:03.757959 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84,"mutex_wait_us":2}
I20260811 18:55:03.758139 31386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:03.753908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.130:54621 (local address 127.27.180.131:46825)
0811 18:55:03.754032 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:03.754037 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:03.754054 (+ 17us) server_negotiation.cc:407] Connection header received
0811 18:55:03.754172 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:03.754175 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:03.754227 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:03.754316 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:55:03.754610 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:03.755495 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:03.757120 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:03.757608 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:03.757746 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:03.757801 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:03.757935 (+ 134us) server_negotiation.cc:299] Negotiation successful
0811 18:55:03.758006 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 18:55:03.758711 31275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b703eba6157b4964a79e5de799595552" candidate_uuid: "04474360ddcb4b2a8686a22455345733" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76aa1f06e7874cde9accdecc25c56921" is_pre_election: true
I20260811 18:55:03.758774 31404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:03.753783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.129:38053 (local address 127.27.180.130:35351)
0811 18:55:03.754175 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:03.754186 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:03.754264 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:03.754525 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:03.754528 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:03.754545 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:55:03.754656 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:03.754661 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:03.755609 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:03.755612 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:03.757356 (+ 1744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:03.757366 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:03.758373 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:03.758377 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:03.758484 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:03.758504 (+ 20us) client_negotiation.cc:769] Sending connection context
0811 18:55:03.758569 (+ 65us) client_negotiation.cc:240] Negotiation successful
0811 18:55:03.758636 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"mutex_wait_us":19,"thread_start_us":150,"threads_started":1}
I20260811 18:55:03.758801 31384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:03.753987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.130:35351 (local address 127.27.180.129:38053)
0811 18:55:03.754161 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:03.754165 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:03.754287 (+ 122us) server_negotiation.cc:407] Connection header received
0811 18:55:03.754388 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:03.754391 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:03.754440 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:03.754500 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:55:03.754759 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:03.755498 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:03.757496 (+ 1998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:03.758120 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:03.758250 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:03.758301 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:03.758612 (+ 311us) server_negotiation.cc:299] Negotiation successful
0811 18:55:03.758696 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260811 18:55:03.759039 31275 raft_consensus.cc:2454] T b703eba6157b4964a79e5de799595552 P 76aa1f06e7874cde9accdecc25c56921 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 04474360ddcb4b2a8686a22455345733 in term 0.
I20260811 18:55:03.759457 30759 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b703eba6157b4964a79e5de799595552" candidate_uuid: "04474360ddcb4b2a8686a22455345733" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76b321a2e8c3434a8a225f70d4b24567" is_pre_election: true
I20260811 18:55:03.759559 30887 leader_election.cc:302] T b703eba6157b4964a79e5de799595552 P 04474360ddcb4b2a8686a22455345733 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04474360ddcb4b2a8686a22455345733, 76aa1f06e7874cde9accdecc25c56921; no voters:
I20260811 18:55:03.759759 30759 raft_consensus.cc:2454] T b703eba6157b4964a79e5de799595552 P 76b321a2e8c3434a8a225f70d4b24567 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 04474360ddcb4b2a8686a22455345733 in term 0.
I20260811 18:55:03.759856 31400 raft_consensus.cc:2781] T b703eba6157b4964a79e5de799595552 P 04474360ddcb4b2a8686a22455345733 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:55:03.759953 31400 raft_consensus.cc:482] T b703eba6157b4964a79e5de799595552 P 04474360ddcb4b2a8686a22455345733 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:55:03.760008 31400 raft_consensus.cc:3037] T b703eba6157b4964a79e5de799595552 P 04474360ddcb4b2a8686a22455345733 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:55:03.761065 31400 raft_consensus.cc:504] T b703eba6157b4964a79e5de799595552 P 04474360ddcb4b2a8686a22455345733 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04474360ddcb4b2a8686a22455345733" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 43235 } } peers { permanent_uuid: "76aa1f06e7874cde9accdecc25c56921" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 46825 } } peers { permanent_uuid: "76b321a2e8c3434a8a225f70d4b24567" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38053 } }
I20260811 18:55:03.761417 31400 leader_election.cc:288] T b703eba6157b4964a79e5de799595552 P 04474360ddcb4b2a8686a22455345733 [CANDIDATE]: Term 1 election: Requested vote from peers 76aa1f06e7874cde9accdecc25c56921 (127.27.180.131:46825), 76b321a2e8c3434a8a225f70d4b24567 (127.27.180.129:38053)
I20260811 18:55:03.761771 30759 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b703eba6157b4964a79e5de799595552" candidate_uuid: "04474360ddcb4b2a8686a22455345733" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76b321a2e8c3434a8a225f70d4b24567"
I20260811 18:55:03.761881 30759 raft_consensus.cc:3037] T b703eba6157b4964a79e5de799595552 P 76b321a2e8c3434a8a225f70d4b24567 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:55:03.762939 30759 raft_consensus.cc:2454] T b703eba6157b4964a79e5de799595552 P 76b321a2e8c3434a8a225f70d4b24567 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 04474360ddcb4b2a8686a22455345733 in term 1.
I20260811 18:55:03.763199 31275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b703eba6157b4964a79e5de799595552" candidate_uuid: "04474360ddcb4b2a8686a22455345733" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76aa1f06e7874cde9accdecc25c56921"
I20260811 18:55:03.763262 30890 leader_election.cc:302] T b703eba6157b4964a79e5de799595552 P 04474360ddcb4b2a8686a22455345733 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04474360ddcb4b2a8686a22455345733, 76b321a2e8c3434a8a225f70d4b24567; no voters:
I20260811 18:55:03.763315 31275 raft_consensus.cc:3037] T b703eba6157b4964a79e5de799595552 P 76aa1f06e7874cde9accdecc25c56921 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:55:03.763470 31400 raft_consensus.cc:2781] T b703eba6157b4964a79e5de799595552 P 04474360ddcb4b2a8686a22455345733 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:55:03.763731 31400 raft_consensus.cc:686] T b703eba6157b4964a79e5de799595552 P 04474360ddcb4b2a8686a22455345733 [term 1 LEADER]: Becoming Leader. State: Replica: 04474360ddcb4b2a8686a22455345733, State: Running, Role: LEADER
I20260811 18:55:03.764070 31400 consensus_queue.cc:237] T b703eba6157b4964a79e5de799595552 P 04474360ddcb4b2a8686a22455345733 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04474360ddcb4b2a8686a22455345733" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 43235 } } peers { permanent_uuid: "76aa1f06e7874cde9accdecc25c56921" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 46825 } } peers { permanent_uuid: "76b321a2e8c3434a8a225f70d4b24567" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38053 } }
I20260811 18:55:03.765127 31275 raft_consensus.cc:2454] T b703eba6157b4964a79e5de799595552 P 76aa1f06e7874cde9accdecc25c56921 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 04474360ddcb4b2a8686a22455345733 in term 1.
I20260811 18:55:03.767465 30477 catalog_manager.cc:5495] T b703eba6157b4964a79e5de799595552 P 04474360ddcb4b2a8686a22455345733 reported cstate change: term changed from 0 to 1, leader changed from <none> to 04474360ddcb4b2a8686a22455345733 (127.27.180.130). New cstate: current_term: 1 leader_uuid: "04474360ddcb4b2a8686a22455345733" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04474360ddcb4b2a8686a22455345733" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 43235 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "76aa1f06e7874cde9accdecc25c56921" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 46825 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76b321a2e8c3434a8a225f70d4b24567" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38053 } health_report { overall_health: UNKNOWN } } }
I20260811 18:55:03.785969 31380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:03.782504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37602 (local address 127.27.180.130:43235)
0811 18:55:03.782612 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:03.782616 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:03.782672 (+ 56us) server_negotiation.cc:407] Connection header received
0811 18:55:03.782868 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:03.782871 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:03.782921 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:03.782998 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:55:03.783200 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:03.783676 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:03.784504 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:03.784708 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:03.784779 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:03.785048 (+ 269us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:55:03.785507 (+ 459us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:55:03.785615 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:03.785767 (+ 152us) server_negotiation.cc:299] Negotiation successful
0811 18:55:03.785840 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 18:55:03.792006 30759 raft_consensus.cc:1261] T b703eba6157b4964a79e5de799595552 P 76b321a2e8c3434a8a225f70d4b24567 [term 1 FOLLOWER]: Refusing update from remote peer 04474360ddcb4b2a8686a22455345733: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:55:03.792730 31400 consensus_queue.cc:1035] T b703eba6157b4964a79e5de799595552 P 04474360ddcb4b2a8686a22455345733 [LEADER]: Connected to new peer: Peer: permanent_uuid: "76b321a2e8c3434a8a225f70d4b24567" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:55:03.793721 31275 raft_consensus.cc:1261] T b703eba6157b4964a79e5de799595552 P 76aa1f06e7874cde9accdecc25c56921 [term 1 FOLLOWER]: Refusing update from remote peer 04474360ddcb4b2a8686a22455345733: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:55:03.794277 31400 consensus_queue.cc:1035] T b703eba6157b4964a79e5de799595552 P 04474360ddcb4b2a8686a22455345733 [LEADER]: Connected to new peer: Peer: permanent_uuid: "76aa1f06e7874cde9accdecc25c56921" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 46825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:55:03.800287 31409 mvcc.cc:205] Tried to move back new op lower bound from 7317399567524016128 to 7317399567420923904. Current Snapshot: MvccSnapshot[applied={T|T < 7317399567524016128}]
I20260811 18:55:03.801131 31411 mvcc.cc:205] Tried to move back new op lower bound from 7317399567524016128 to 7317399567420923904. Current Snapshot: MvccSnapshot[applied={T|T < 7317399567524016128 or (T in {7317399567524016128})}]
I20260811 18:55:03.813735 30477 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50778:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:55:03.814829 30477 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50778:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:55:03.817540 30477 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ad3417ccb7ca4db993c61829c1f3bdbe: Sending DeleteTablet for 3 replicas of tablet b703eba6157b4964a79e5de799595552
I20260811 18:55:03.818392 31234 tablet_service.cc:1514] Processing DeleteTablet for tablet b703eba6157b4964a79e5de799595552 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:55:03 UTC) from {username='kudu', principal='kudu/127.27.180.190@KRBTEST.COM'} at 127.0.0.1:35480
I20260811 18:55:03.818382 30726 tablet_service.cc:1514] Processing DeleteTablet for tablet b703eba6157b4964a79e5de799595552 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:55:03 UTC) from {username='kudu', principal='kudu/127.27.180.190@KRBTEST.COM'} at 127.0.0.1:37806
I20260811 18:55:03.818554 30974 tablet_service.cc:1514] Processing DeleteTablet for tablet b703eba6157b4964a79e5de799595552 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:55:03 UTC) from {username='kudu', principal='kudu/127.27.180.190@KRBTEST.COM'} at 127.0.0.1:37596
I20260811 18:55:03.819018 31425 tablet_replica.cc:330] T b703eba6157b4964a79e5de799595552 P 76b321a2e8c3434a8a225f70d4b24567: stopping tablet replica
I20260811 18:55:03.819300 31425 raft_consensus.cc:2229] T b703eba6157b4964a79e5de799595552 P 76b321a2e8c3434a8a225f70d4b24567 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:55:03.819582 31425 raft_consensus.cc:2258] T b703eba6157b4964a79e5de799595552 P 76b321a2e8c3434a8a225f70d4b24567 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:55:03.819756 30477 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50778:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:55:03.820147 30477 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:55:03.820518 31422 tablet_replica.cc:330] T b703eba6157b4964a79e5de799595552 P 76aa1f06e7874cde9accdecc25c56921: stopping tablet replica
I20260811 18:55:03.820791 31422 raft_consensus.cc:2229] T b703eba6157b4964a79e5de799595552 P 76aa1f06e7874cde9accdecc25c56921 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:55:03.820935 31426 tablet_replica.cc:330] T b703eba6157b4964a79e5de799595552 P 04474360ddcb4b2a8686a22455345733: stopping tablet replica
I20260811 18:55:03.821059 31422 raft_consensus.cc:2258] T b703eba6157b4964a79e5de799595552 P 76aa1f06e7874cde9accdecc25c56921 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:55:03.821206 31426 raft_consensus.cc:2229] T b703eba6157b4964a79e5de799595552 P 04474360ddcb4b2a8686a22455345733 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:55:03.821585 31426 raft_consensus.cc:2258] T b703eba6157b4964a79e5de799595552 P 04474360ddcb4b2a8686a22455345733 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:55:03.822867 31422 ts_tablet_manager.cc:1905] T b703eba6157b4964a79e5de799595552 P 76aa1f06e7874cde9accdecc25c56921: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:55:03.822928 31425 ts_tablet_manager.cc:1905] T b703eba6157b4964a79e5de799595552 P 76b321a2e8c3434a8a225f70d4b24567: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:55:03.825657 31425 ts_tablet_manager.cc:1918] T b703eba6157b4964a79e5de799595552 P 76b321a2e8c3434a8a225f70d4b24567: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:55:03.825747 31425 log.cc:1196] T b703eba6157b4964a79e5de799595552 P 76b321a2e8c3434a8a225f70d4b24567: Deleting WAL directory at /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/minicluster-data/ts-0/wal/wals/b703eba6157b4964a79e5de799595552
I20260811 18:55:03.825987 31425 ts_tablet_manager.cc:1939] T b703eba6157b4964a79e5de799595552 P 76b321a2e8c3434a8a225f70d4b24567: Deleting consensus metadata
I20260811 18:55:03.826841 30726 tablet_service.cc:1467] Processing CreateTablet for tablet f7aa1cf8518c43689302ccaff8dd2416 (DEFAULT_TABLE table=test-table [id=f4756b7337db4d1d9ddd5108ddce22cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:55:03.827033 30726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f7aa1cf8518c43689302ccaff8dd2416. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:55:03.827270 30974 tablet_service.cc:1467] Processing CreateTablet for tablet f7aa1cf8518c43689302ccaff8dd2416 (DEFAULT_TABLE table=test-table [id=f4756b7337db4d1d9ddd5108ddce22cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:55:03.827499 31422 ts_tablet_manager.cc:1918] T b703eba6157b4964a79e5de799595552 P 76aa1f06e7874cde9accdecc25c56921: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:55:03.827531 30974 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f7aa1cf8518c43689302ccaff8dd2416. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:55:03.827749 31422 log.cc:1196] T b703eba6157b4964a79e5de799595552 P 76aa1f06e7874cde9accdecc25c56921: Deleting WAL directory at /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/minicluster-data/ts-2/wal/wals/b703eba6157b4964a79e5de799595552
I20260811 18:55:03.828130 31422 ts_tablet_manager.cc:1939] T b703eba6157b4964a79e5de799595552 P 76aa1f06e7874cde9accdecc25c56921: Deleting consensus metadata
I20260811 18:55:03.829188 31390 tablet_bootstrap.cc:492] T f7aa1cf8518c43689302ccaff8dd2416 P 76b321a2e8c3434a8a225f70d4b24567: Bootstrap starting.
I20260811 18:55:03.829403 30425 catalog_manager.cc:4841] TS 76aa1f06e7874cde9accdecc25c56921 (127.27.180.131:46825): tablet b703eba6157b4964a79e5de799595552 (table test-table [id=a36f1d3ff01547acbaf0b57c759f3168]) successfully deleted
I20260811 18:55:03.829893 31391 tablet_bootstrap.cc:492] T f7aa1cf8518c43689302ccaff8dd2416 P 04474360ddcb4b2a8686a22455345733: Bootstrap starting.
I20260811 18:55:03.830098 31234 tablet_service.cc:1467] Processing CreateTablet for tablet f7aa1cf8518c43689302ccaff8dd2416 (DEFAULT_TABLE table=test-table [id=f4756b7337db4d1d9ddd5108ddce22cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:55:03.830212 31390 tablet_bootstrap.cc:654] T f7aa1cf8518c43689302ccaff8dd2416 P 76b321a2e8c3434a8a225f70d4b24567: Neither blocks nor log segments found. Creating new log.
I20260811 18:55:03.830361 31234 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f7aa1cf8518c43689302ccaff8dd2416. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:55:03.830804 31391 tablet_bootstrap.cc:654] T f7aa1cf8518c43689302ccaff8dd2416 P 04474360ddcb4b2a8686a22455345733: Neither blocks nor log segments found. Creating new log.
I20260811 18:55:03.831642 31390 tablet_bootstrap.cc:492] T f7aa1cf8518c43689302ccaff8dd2416 P 76b321a2e8c3434a8a225f70d4b24567: No bootstrap required, opened a new log
I20260811 18:55:03.831735 31390 ts_tablet_manager.cc:1397] T f7aa1cf8518c43689302ccaff8dd2416 P 76b321a2e8c3434a8a225f70d4b24567: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 18:55:03.831979 31426 ts_tablet_manager.cc:1905] T b703eba6157b4964a79e5de799595552 P 04474360ddcb4b2a8686a22455345733: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:55:03.832495 31390 raft_consensus.cc:348] T f7aa1cf8518c43689302ccaff8dd2416 P 76b321a2e8c3434a8a225f70d4b24567 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76b321a2e8c3434a8a225f70d4b24567" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38053 } } peers { permanent_uuid: "04474360ddcb4b2a8686a22455345733" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 43235 } } peers { permanent_uuid: "76aa1f06e7874cde9accdecc25c56921" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 46825 } }
I20260811 18:55:03.832624 31390 raft_consensus.cc:374] T f7aa1cf8518c43689302ccaff8dd2416 P 76b321a2e8c3434a8a225f70d4b24567 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:55:03.832664 31390 raft_consensus.cc:729] T f7aa1cf8518c43689302ccaff8dd2416 P 76b321a2e8c3434a8a225f70d4b24567 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76b321a2e8c3434a8a225f70d4b24567, State: Initialized, Role: FOLLOWER
I20260811 18:55:03.832787 31390 consensus_queue.cc:260] T f7aa1cf8518c43689302ccaff8dd2416 P 76b321a2e8c3434a8a225f70d4b24567 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76b321a2e8c3434a8a225f70d4b24567" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38053 } } peers { permanent_uuid: "04474360ddcb4b2a8686a22455345733" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 43235 } } peers { permanent_uuid: "76aa1f06e7874cde9accdecc25c56921" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 46825 } }
I20260811 18:55:03.833069 31390 ts_tablet_manager.cc:1428] T f7aa1cf8518c43689302ccaff8dd2416 P 76b321a2e8c3434a8a225f70d4b24567: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 18:55:03.833215 31387 tablet_bootstrap.cc:492] T f7aa1cf8518c43689302ccaff8dd2416 P 76aa1f06e7874cde9accdecc25c56921: Bootstrap starting.
I20260811 18:55:03.834144 31387 tablet_bootstrap.cc:654] T f7aa1cf8518c43689302ccaff8dd2416 P 76aa1f06e7874cde9accdecc25c56921: Neither blocks nor log segments found. Creating new log.
I20260811 18:55:03.835489 31391 tablet_bootstrap.cc:492] T f7aa1cf8518c43689302ccaff8dd2416 P 04474360ddcb4b2a8686a22455345733: No bootstrap required, opened a new log
I20260811 18:55:03.835593 31391 ts_tablet_manager.cc:1397] T f7aa1cf8518c43689302ccaff8dd2416 P 04474360ddcb4b2a8686a22455345733: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.001s
I20260811 18:55:03.835986 31391 raft_consensus.cc:348] T f7aa1cf8518c43689302ccaff8dd2416 P 04474360ddcb4b2a8686a22455345733 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76b321a2e8c3434a8a225f70d4b24567" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38053 } } peers { permanent_uuid: "04474360ddcb4b2a8686a22455345733" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 43235 } } peers { permanent_uuid: "76aa1f06e7874cde9accdecc25c56921" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 46825 } }
I20260811 18:55:03.836076 31391 raft_consensus.cc:374] T f7aa1cf8518c43689302ccaff8dd2416 P 04474360ddcb4b2a8686a22455345733 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:55:03.836097 31391 raft_consensus.cc:729] T f7aa1cf8518c43689302ccaff8dd2416 P 04474360ddcb4b2a8686a22455345733 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04474360ddcb4b2a8686a22455345733, State: Initialized, Role: FOLLOWER
I20260811 18:55:03.836193 31391 consensus_queue.cc:260] T f7aa1cf8518c43689302ccaff8dd2416 P 04474360ddcb4b2a8686a22455345733 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76b321a2e8c3434a8a225f70d4b24567" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38053 } } peers { permanent_uuid: "04474360ddcb4b2a8686a22455345733" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 43235 } } peers { permanent_uuid: "76aa1f06e7874cde9accdecc25c56921" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 46825 } }
I20260811 18:55:03.836431 31391 ts_tablet_manager.cc:1428] T f7aa1cf8518c43689302ccaff8dd2416 P 04474360ddcb4b2a8686a22455345733: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 18:55:03.836997 31426 ts_tablet_manager.cc:1918] T b703eba6157b4964a79e5de799595552 P 04474360ddcb4b2a8686a22455345733: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:55:03.838074 31426 log.cc:1196] T b703eba6157b4964a79e5de799595552 P 04474360ddcb4b2a8686a22455345733: Deleting WAL directory at /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/minicluster-data/ts-1/wal/wals/b703eba6157b4964a79e5de799595552
I20260811 18:55:03.838363 31426 ts_tablet_manager.cc:1939] T b703eba6157b4964a79e5de799595552 P 04474360ddcb4b2a8686a22455345733: Deleting consensus metadata
I20260811 18:55:03.839236 30425 catalog_manager.cc:4841] TS 04474360ddcb4b2a8686a22455345733 (127.27.180.130:43235): tablet b703eba6157b4964a79e5de799595552 (table test-table [id=a36f1d3ff01547acbaf0b57c759f3168]) successfully deleted
I20260811 18:55:03.839320 30427 catalog_manager.cc:4841] TS 76b321a2e8c3434a8a225f70d4b24567 (127.27.180.129:38053): tablet b703eba6157b4964a79e5de799595552 (table test-table [id=a36f1d3ff01547acbaf0b57c759f3168]) successfully deleted
I20260811 18:55:03.840101 31387 tablet_bootstrap.cc:492] T f7aa1cf8518c43689302ccaff8dd2416 P 76aa1f06e7874cde9accdecc25c56921: No bootstrap required, opened a new log
I20260811 18:55:03.840179 31387 ts_tablet_manager.cc:1397] T f7aa1cf8518c43689302ccaff8dd2416 P 76aa1f06e7874cde9accdecc25c56921: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260811 18:55:03.840540 31387 raft_consensus.cc:348] T f7aa1cf8518c43689302ccaff8dd2416 P 76aa1f06e7874cde9accdecc25c56921 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76b321a2e8c3434a8a225f70d4b24567" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38053 } } peers { permanent_uuid: "04474360ddcb4b2a8686a22455345733" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 43235 } } peers { permanent_uuid: "76aa1f06e7874cde9accdecc25c56921" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 46825 } }
I20260811 18:55:03.840631 31387 raft_consensus.cc:374] T f7aa1cf8518c43689302ccaff8dd2416 P 76aa1f06e7874cde9accdecc25c56921 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:55:03.840659 31387 raft_consensus.cc:729] T f7aa1cf8518c43689302ccaff8dd2416 P 76aa1f06e7874cde9accdecc25c56921 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76aa1f06e7874cde9accdecc25c56921, State: Initialized, Role: FOLLOWER
I20260811 18:55:03.840773 31387 consensus_queue.cc:260] T f7aa1cf8518c43689302ccaff8dd2416 P 76aa1f06e7874cde9accdecc25c56921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76b321a2e8c3434a8a225f70d4b24567" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38053 } } peers { permanent_uuid: "04474360ddcb4b2a8686a22455345733" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 43235 } } peers { permanent_uuid: "76aa1f06e7874cde9accdecc25c56921" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 46825 } }
I20260811 18:55:03.840996 31387 ts_tablet_manager.cc:1428] T f7aa1cf8518c43689302ccaff8dd2416 P 76aa1f06e7874cde9accdecc25c56921: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
W20260811 18:55:03.846583 31108 tablet.cc:2369] T f7aa1cf8518c43689302ccaff8dd2416 P 04474360ddcb4b2a8686a22455345733: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 18:55:03.856580 30837 tablet.cc:2369] T f7aa1cf8518c43689302ccaff8dd2416 P 76b321a2e8c3434a8a225f70d4b24567: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:55:04.039999 31401 raft_consensus.cc:482] T f7aa1cf8518c43689302ccaff8dd2416 P 76b321a2e8c3434a8a225f70d4b24567 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:55:04.040149 31401 raft_consensus.cc:504] T f7aa1cf8518c43689302ccaff8dd2416 P 76b321a2e8c3434a8a225f70d4b24567 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76b321a2e8c3434a8a225f70d4b24567" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38053 } } peers { permanent_uuid: "04474360ddcb4b2a8686a22455345733" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 43235 } } peers { permanent_uuid: "76aa1f06e7874cde9accdecc25c56921" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 46825 } }
I20260811 18:55:04.041322 31401 leader_election.cc:288] T f7aa1cf8518c43689302ccaff8dd2416 P 76b321a2e8c3434a8a225f70d4b24567 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 04474360ddcb4b2a8686a22455345733 (127.27.180.130:43235), 76aa1f06e7874cde9accdecc25c56921 (127.27.180.131:46825)
I20260811 18:55:04.045456 31380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:04.041696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.129:57691 (local address 127.27.180.130:43235)
0811 18:55:04.041868 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:04.041873 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:04.041992 (+ 119us) server_negotiation.cc:407] Connection header received
0811 18:55:04.042228 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:04.042232 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:04.042305 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:04.042376 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:55:04.042697 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:04.043286 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:04.044447 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:04.044853 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:04.044946 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:04.045001 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:04.045244 (+ 243us) server_negotiation.cc:299] Negotiation successful
0811 18:55:04.045330 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 18:55:04.045456 31386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:04.042062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.129:36017 (local address 127.27.180.131:46825)
0811 18:55:04.042215 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:04.042220 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:04.042349 (+ 129us) server_negotiation.cc:407] Connection header received
0811 18:55:04.042506 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:04.042510 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:04.042558 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:04.042632 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:55:04.042942 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:04.043462 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:04.044490 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:04.044896 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:04.045019 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:04.045078 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:04.045262 (+ 184us) server_negotiation.cc:299] Negotiation successful
0811 18:55:04.045329 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 18:55:04.045490 31437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:04.041938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.131:46825 (local address 127.27.180.129:36017)
0811 18:55:04.042260 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:04.042274 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:04.042374 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:04.042654 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:04.042656 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:04.042676 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:55:04.042827 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:04.042834 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:04.043570 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:04.043573 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:04.044357 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:04.044367 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:04.045025 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:04.045029 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:04.045143 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:04.045164 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 18:55:04.045234 (+ 70us) client_negotiation.cc:240] Negotiation successful
0811 18:55:04.045330 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"mutex_wait_us":21,"thread_start_us":93,"threads_started":1}
I20260811 18:55:04.045483 31436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:04.041569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.130:43235 (local address 127.27.180.129:57691)
0811 18:55:04.041886 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:04.041905 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:04.042077 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:04.042425 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:04.042428 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:04.042451 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:55:04.042586 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:04.042593 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:04.043403 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:04.043406 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:04.044321 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:04.044330 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:04.044989 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:04.044994 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:04.045116 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:04.045145 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 18:55:04.045218 (+ 73us) client_negotiation.cc:240] Negotiation successful
0811 18:55:04.045306 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":228,"mutex_wait_us":22,"thread_start_us":83,"threads_started":1}
I20260811 18:55:04.046052 31275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7aa1cf8518c43689302ccaff8dd2416" candidate_uuid: "76b321a2e8c3434a8a225f70d4b24567" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76aa1f06e7874cde9accdecc25c56921" is_pre_election: true
I20260811 18:55:04.046100 31015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7aa1cf8518c43689302ccaff8dd2416" candidate_uuid: "76b321a2e8c3434a8a225f70d4b24567" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04474360ddcb4b2a8686a22455345733" is_pre_election: true
I20260811 18:55:04.046221 31275 raft_consensus.cc:2454] T f7aa1cf8518c43689302ccaff8dd2416 P 76aa1f06e7874cde9accdecc25c56921 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76b321a2e8c3434a8a225f70d4b24567 in term 0.
I20260811 18:55:04.046322 31015 raft_consensus.cc:2454] T f7aa1cf8518c43689302ccaff8dd2416 P 04474360ddcb4b2a8686a22455345733 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76b321a2e8c3434a8a225f70d4b24567 in term 0.
I20260811 18:55:04.046602 30627 leader_election.cc:302] T f7aa1cf8518c43689302ccaff8dd2416 P 76b321a2e8c3434a8a225f70d4b24567 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76aa1f06e7874cde9accdecc25c56921, 76b321a2e8c3434a8a225f70d4b24567; no voters:
I20260811 18:55:04.046861 31401 raft_consensus.cc:2781] T f7aa1cf8518c43689302ccaff8dd2416 P 76b321a2e8c3434a8a225f70d4b24567 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:55:04.046926 31401 raft_consensus.cc:482] T f7aa1cf8518c43689302ccaff8dd2416 P 76b321a2e8c3434a8a225f70d4b24567 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:55:04.046968 31401 raft_consensus.cc:3037] T f7aa1cf8518c43689302ccaff8dd2416 P 76b321a2e8c3434a8a225f70d4b24567 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:55:04.047966 31401 raft_consensus.cc:504] T f7aa1cf8518c43689302ccaff8dd2416 P 76b321a2e8c3434a8a225f70d4b24567 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76b321a2e8c3434a8a225f70d4b24567" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38053 } } peers { permanent_uuid: "04474360ddcb4b2a8686a22455345733" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 43235 } } peers { permanent_uuid: "76aa1f06e7874cde9accdecc25c56921" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 46825 } }
I20260811 18:55:04.048334 31401 leader_election.cc:288] T f7aa1cf8518c43689302ccaff8dd2416 P 76b321a2e8c3434a8a225f70d4b24567 [CANDIDATE]: Term 1 election: Requested vote from peers 04474360ddcb4b2a8686a22455345733 (127.27.180.130:43235), 76aa1f06e7874cde9accdecc25c56921 (127.27.180.131:46825)
I20260811 18:55:04.048698 31015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7aa1cf8518c43689302ccaff8dd2416" candidate_uuid: "76b321a2e8c3434a8a225f70d4b24567" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04474360ddcb4b2a8686a22455345733"
I20260811 18:55:04.048698 31275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7aa1cf8518c43689302ccaff8dd2416" candidate_uuid: "76b321a2e8c3434a8a225f70d4b24567" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76aa1f06e7874cde9accdecc25c56921"
I20260811 18:55:04.048825 31015 raft_consensus.cc:3037] T f7aa1cf8518c43689302ccaff8dd2416 P 04474360ddcb4b2a8686a22455345733 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:55:04.048821 31275 raft_consensus.cc:3037] T f7aa1cf8518c43689302ccaff8dd2416 P 76aa1f06e7874cde9accdecc25c56921 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:55:04.049933 31275 raft_consensus.cc:2454] T f7aa1cf8518c43689302ccaff8dd2416 P 76aa1f06e7874cde9accdecc25c56921 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76b321a2e8c3434a8a225f70d4b24567 in term 1.
I20260811 18:55:04.049933 31015 raft_consensus.cc:2454] T f7aa1cf8518c43689302ccaff8dd2416 P 04474360ddcb4b2a8686a22455345733 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76b321a2e8c3434a8a225f70d4b24567 in term 1.
I20260811 18:55:04.050205 30627 leader_election.cc:302] T f7aa1cf8518c43689302ccaff8dd2416 P 76b321a2e8c3434a8a225f70d4b24567 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76aa1f06e7874cde9accdecc25c56921, 76b321a2e8c3434a8a225f70d4b24567; no voters:
I20260811 18:55:04.050376 31401 raft_consensus.cc:2781] T f7aa1cf8518c43689302ccaff8dd2416 P 76b321a2e8c3434a8a225f70d4b24567 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:55:04.050580 31401 raft_consensus.cc:686] T f7aa1cf8518c43689302ccaff8dd2416 P 76b321a2e8c3434a8a225f70d4b24567 [term 1 LEADER]: Becoming Leader. State: Replica: 76b321a2e8c3434a8a225f70d4b24567, State: Running, Role: LEADER
I20260811 18:55:04.050797 31401 consensus_queue.cc:237] T f7aa1cf8518c43689302ccaff8dd2416 P 76b321a2e8c3434a8a225f70d4b24567 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76b321a2e8c3434a8a225f70d4b24567" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38053 } } peers { permanent_uuid: "04474360ddcb4b2a8686a22455345733" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 43235 } } peers { permanent_uuid: "76aa1f06e7874cde9accdecc25c56921" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 46825 } }
I20260811 18:55:04.052938 30477 catalog_manager.cc:5495] T f7aa1cf8518c43689302ccaff8dd2416 P 76b321a2e8c3434a8a225f70d4b24567 reported cstate change: term changed from 0 to 1, leader changed from <none> to 76b321a2e8c3434a8a225f70d4b24567 (127.27.180.129). New cstate: current_term: 1 leader_uuid: "76b321a2e8c3434a8a225f70d4b24567" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76b321a2e8c3434a8a225f70d4b24567" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38053 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "04474360ddcb4b2a8686a22455345733" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 43235 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76aa1f06e7874cde9accdecc25c56921" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 46825 } health_report { overall_health: UNKNOWN } } }
W20260811 18:55:04.066533 31353 tablet.cc:2369] T f7aa1cf8518c43689302ccaff8dd2416 P 76aa1f06e7874cde9accdecc25c56921: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 11 18:55:04 dist-test-slave-2j7r krb5kdc[30388](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474502, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.190@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM
I20260811 18:55:04.102763 31446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:04.094667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:34039 (local address 127.0.0.1:50782)
0811 18:55:04.094958 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:04.094970 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:04.095075 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:04.095299 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:04.095301 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:04.095717 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:04.095824 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:04.095831 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:04.096555 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:04.096559 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:04.097158 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:04.097168 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:04.097572 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:04.097575 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:04.097704 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:04.098191 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:04.098202 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:04.099657 (+ 1455us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:55:04.101275 (+ 1618us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:04.101285 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:04.101292 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:04.101517 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:04.101770 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:04.101773 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:04.101775 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:04.101860 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:04.102207 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:55:04.102213 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:55:04.102322 (+ 109us) client_negotiation.cc:769] Sending connection context
0811 18:55:04.102418 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 18:55:04.102571 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215,"mutex_wait_us":19,"thread_start_us":108,"threads_started":1}
I20260811 18:55:04.102762 30563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:04.094751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50782 (local address 127.27.180.190:34039)
0811 18:55:04.094887 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:04.094892 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:04.095041 (+ 149us) server_negotiation.cc:407] Connection header received
0811 18:55:04.095160 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:04.095163 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:04.095203 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:04.095266 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:04.095942 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:04.096454 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:04.097261 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:04.097475 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:04.097561 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:04.099785 (+ 2224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:04.099789 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:04.099791 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:04.099816 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:04.101173 (+ 1357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:04.101608 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:04.101612 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:04.101614 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:04.101690 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:04.101939 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:04.101942 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:04.101944 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:04.102108 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:04.102193 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:04.102479 (+ 286us) server_negotiation.cc:299] Negotiation successful
0811 18:55:04.102621 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":82,"server-negotiator.queue_time_us":49}
I20260811 18:55:04.106827 30477 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.27.180.190@KRBTEST.COM'} at 127.0.0.1:50782:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 18:55:04.107545 30477 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:55:04.112151 30726 tablet_service.cc:1467] Processing CreateTablet for tablet 2449b162945f4d9991293b63c2babc50 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=83dba803bbe9498db8ce5b310fd90a49]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 18:55:04.112432 30974 tablet_service.cc:1467] Processing CreateTablet for tablet 2449b162945f4d9991293b63c2babc50 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=83dba803bbe9498db8ce5b310fd90a49]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 18:55:04.112581 30726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2449b162945f4d9991293b63c2babc50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:55:04.112768 30974 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2449b162945f4d9991293b63c2babc50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:55:04.115489 31391 tablet_bootstrap.cc:492] T 2449b162945f4d9991293b63c2babc50 P 04474360ddcb4b2a8686a22455345733: Bootstrap starting.
I20260811 18:55:04.115790 31234 tablet_service.cc:1467] Processing CreateTablet for tablet 2449b162945f4d9991293b63c2babc50 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=83dba803bbe9498db8ce5b310fd90a49]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 18:55:04.116065 31234 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2449b162945f4d9991293b63c2babc50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:55:04.116328 31391 tablet_bootstrap.cc:654] T 2449b162945f4d9991293b63c2babc50 P 04474360ddcb4b2a8686a22455345733: Neither blocks nor log segments found. Creating new log.
I20260811 18:55:04.117460 31390 tablet_bootstrap.cc:492] T 2449b162945f4d9991293b63c2babc50 P 76b321a2e8c3434a8a225f70d4b24567: Bootstrap starting.
I20260811 18:55:04.118244 31390 tablet_bootstrap.cc:654] T 2449b162945f4d9991293b63c2babc50 P 76b321a2e8c3434a8a225f70d4b24567: Neither blocks nor log segments found. Creating new log.
I20260811 18:55:04.118604 31387 tablet_bootstrap.cc:492] T 2449b162945f4d9991293b63c2babc50 P 76aa1f06e7874cde9accdecc25c56921: Bootstrap starting.
I20260811 18:55:04.119619 31387 tablet_bootstrap.cc:654] T 2449b162945f4d9991293b63c2babc50 P 76aa1f06e7874cde9accdecc25c56921: Neither blocks nor log segments found. Creating new log.
I20260811 18:55:04.119933 31390 tablet_bootstrap.cc:492] T 2449b162945f4d9991293b63c2babc50 P 76b321a2e8c3434a8a225f70d4b24567: No bootstrap required, opened a new log
I20260811 18:55:04.120026 31390 ts_tablet_manager.cc:1397] T 2449b162945f4d9991293b63c2babc50 P 76b321a2e8c3434a8a225f70d4b24567: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260811 18:55:04.120514 31391 tablet_bootstrap.cc:492] T 2449b162945f4d9991293b63c2babc50 P 04474360ddcb4b2a8686a22455345733: No bootstrap required, opened a new log
I20260811 18:55:04.120478 31390 raft_consensus.cc:348] T 2449b162945f4d9991293b63c2babc50 P 76b321a2e8c3434a8a225f70d4b24567 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76b321a2e8c3434a8a225f70d4b24567" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38053 } } peers { permanent_uuid: "04474360ddcb4b2a8686a22455345733" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 43235 } } peers { permanent_uuid: "76aa1f06e7874cde9accdecc25c56921" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 46825 } }
I20260811 18:55:04.120582 31390 raft_consensus.cc:374] T 2449b162945f4d9991293b63c2babc50 P 76b321a2e8c3434a8a225f70d4b24567 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:55:04.120604 31391 ts_tablet_manager.cc:1397] T 2449b162945f4d9991293b63c2babc50 P 04474360ddcb4b2a8686a22455345733: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.001s
I20260811 18:55:04.120622 31390 raft_consensus.cc:729] T 2449b162945f4d9991293b63c2babc50 P 76b321a2e8c3434a8a225f70d4b24567 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76b321a2e8c3434a8a225f70d4b24567, State: Initialized, Role: FOLLOWER
I20260811 18:55:04.120754 31390 consensus_queue.cc:260] T 2449b162945f4d9991293b63c2babc50 P 76b321a2e8c3434a8a225f70d4b24567 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76b321a2e8c3434a8a225f70d4b24567" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38053 } } peers { permanent_uuid: "04474360ddcb4b2a8686a22455345733" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 43235 } } peers { permanent_uuid: "76aa1f06e7874cde9accdecc25c56921" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 46825 } }
I20260811 18:55:04.121030 31390 ts_tablet_manager.cc:1428] T 2449b162945f4d9991293b63c2babc50 P 76b321a2e8c3434a8a225f70d4b24567: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 18:55:04.121002 31391 raft_consensus.cc:348] T 2449b162945f4d9991293b63c2babc50 P 04474360ddcb4b2a8686a22455345733 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76b321a2e8c3434a8a225f70d4b24567" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38053 } } peers { permanent_uuid: "04474360ddcb4b2a8686a22455345733" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 43235 } } peers { permanent_uuid: "76aa1f06e7874cde9accdecc25c56921" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 46825 } }
I20260811 18:55:04.121126 31391 raft_consensus.cc:374] T 2449b162945f4d9991293b63c2babc50 P 04474360ddcb4b2a8686a22455345733 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:55:04.121054 31401 tablet_replica.cc:439] T 2449b162945f4d9991293b63c2babc50 P 76b321a2e8c3434a8a225f70d4b24567: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76b321a2e8c3434a8a225f70d4b24567" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38053 } } peers { permanent_uuid: "04474360ddcb4b2a8686a22455345733" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 43235 } } peers { permanent_uuid: "76aa1f06e7874cde9accdecc25c56921" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 46825 } } }
I20260811 18:55:04.121160 31391 raft_consensus.cc:729] T 2449b162945f4d9991293b63c2babc50 P 04474360ddcb4b2a8686a22455345733 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04474360ddcb4b2a8686a22455345733, State: Initialized, Role: FOLLOWER
I20260811 18:55:04.121208 31401 tablet_replica.cc:442] T 2449b162945f4d9991293b63c2babc50 P 76b321a2e8c3434a8a225f70d4b24567: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 18:55:04.121273 31391 consensus_queue.cc:260] T 2449b162945f4d9991293b63c2babc50 P 04474360ddcb4b2a8686a22455345733 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76b321a2e8c3434a8a225f70d4b24567" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38053 } } peers { permanent_uuid: "04474360ddcb4b2a8686a22455345733" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 43235 } } peers { permanent_uuid: "76aa1f06e7874cde9accdecc25c56921" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 46825 } }
I20260811 18:55:04.121547 31391 ts_tablet_manager.cc:1428] T 2449b162945f4d9991293b63c2babc50 P 04474360ddcb4b2a8686a22455345733: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 18:55:04.121797 31400 tablet_replica.cc:439] T 2449b162945f4d9991293b63c2babc50 P 04474360ddcb4b2a8686a22455345733: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76b321a2e8c3434a8a225f70d4b24567" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38053 } } peers { permanent_uuid: "04474360ddcb4b2a8686a22455345733" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 43235 } } peers { permanent_uuid: "76aa1f06e7874cde9accdecc25c56921" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 46825 } } }
I20260811 18:55:04.121917 31400 tablet_replica.cc:442] T 2449b162945f4d9991293b63c2babc50 P 04474360ddcb4b2a8686a22455345733: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 18:55:04.121920 31387 tablet_bootstrap.cc:492] T 2449b162945f4d9991293b63c2babc50 P 76aa1f06e7874cde9accdecc25c56921: No bootstrap required, opened a new log
I20260811 18:55:04.121999 31387 ts_tablet_manager.cc:1397] T 2449b162945f4d9991293b63c2babc50 P 76aa1f06e7874cde9accdecc25c56921: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 18:55:04.122601 31387 raft_consensus.cc:348] T 2449b162945f4d9991293b63c2babc50 P 76aa1f06e7874cde9accdecc25c56921 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76b321a2e8c3434a8a225f70d4b24567" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38053 } } peers { permanent_uuid: "04474360ddcb4b2a8686a22455345733" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 43235 } } peers { permanent_uuid: "76aa1f06e7874cde9accdecc25c56921" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 46825 } }
I20260811 18:55:04.122706 31387 raft_consensus.cc:374] T 2449b162945f4d9991293b63c2babc50 P 76aa1f06e7874cde9accdecc25c56921 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:55:04.122740 31387 raft_consensus.cc:729] T 2449b162945f4d9991293b63c2babc50 P 76aa1f06e7874cde9accdecc25c56921 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76aa1f06e7874cde9accdecc25c56921, State: Initialized, Role: FOLLOWER
I20260811 18:55:04.122859 31387 consensus_queue.cc:260] T 2449b162945f4d9991293b63c2babc50 P 76aa1f06e7874cde9accdecc25c56921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76b321a2e8c3434a8a225f70d4b24567" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38053 } } peers { permanent_uuid: "04474360ddcb4b2a8686a22455345733" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 43235 } } peers { permanent_uuid: "76aa1f06e7874cde9accdecc25c56921" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 46825 } }
I20260811 18:55:04.123297 31387 ts_tablet_manager.cc:1428] T 2449b162945f4d9991293b63c2babc50 P 76aa1f06e7874cde9accdecc25c56921: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 18:55:04.123246 31398 tablet_replica.cc:439] T 2449b162945f4d9991293b63c2babc50 P 76aa1f06e7874cde9accdecc25c56921: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76b321a2e8c3434a8a225f70d4b24567" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38053 } } peers { permanent_uuid: "04474360ddcb4b2a8686a22455345733" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 43235 } } peers { permanent_uuid: "76aa1f06e7874cde9accdecc25c56921" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 46825 } } }
I20260811 18:55:04.123400 31398 tablet_replica.cc:442] T 2449b162945f4d9991293b63c2babc50 P 76aa1f06e7874cde9accdecc25c56921: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 18:55:04.310324 31400 raft_consensus.cc:482] T 2449b162945f4d9991293b63c2babc50 P 04474360ddcb4b2a8686a22455345733 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:55:04.310456 31400 raft_consensus.cc:504] T 2449b162945f4d9991293b63c2babc50 P 04474360ddcb4b2a8686a22455345733 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76b321a2e8c3434a8a225f70d4b24567" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38053 } } peers { permanent_uuid: "04474360ddcb4b2a8686a22455345733" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 43235 } } peers { permanent_uuid: "76aa1f06e7874cde9accdecc25c56921" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 46825 } }
I20260811 18:55:04.310861 31400 leader_election.cc:288] T 2449b162945f4d9991293b63c2babc50 P 04474360ddcb4b2a8686a22455345733 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 76b321a2e8c3434a8a225f70d4b24567 (127.27.180.129:38053), 76aa1f06e7874cde9accdecc25c56921 (127.27.180.131:46825)
I20260811 18:55:04.311285 30759 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2449b162945f4d9991293b63c2babc50" candidate_uuid: "04474360ddcb4b2a8686a22455345733" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76b321a2e8c3434a8a225f70d4b24567" is_pre_election: true
I20260811 18:55:04.311312 31275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2449b162945f4d9991293b63c2babc50" candidate_uuid: "04474360ddcb4b2a8686a22455345733" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76aa1f06e7874cde9accdecc25c56921" is_pre_election: true
I20260811 18:55:04.311427 30759 raft_consensus.cc:2454] T 2449b162945f4d9991293b63c2babc50 P 76b321a2e8c3434a8a225f70d4b24567 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 04474360ddcb4b2a8686a22455345733 in term 0.
I20260811 18:55:04.311476 31275 raft_consensus.cc:2454] T 2449b162945f4d9991293b63c2babc50 P 76aa1f06e7874cde9accdecc25c56921 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 04474360ddcb4b2a8686a22455345733 in term 0.
I20260811 18:55:04.311728 30890 leader_election.cc:302] T 2449b162945f4d9991293b63c2babc50 P 04474360ddcb4b2a8686a22455345733 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04474360ddcb4b2a8686a22455345733, 76b321a2e8c3434a8a225f70d4b24567; no voters:
I20260811 18:55:04.311925 31400 raft_consensus.cc:2781] T 2449b162945f4d9991293b63c2babc50 P 04474360ddcb4b2a8686a22455345733 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:55:04.311992 31400 raft_consensus.cc:482] T 2449b162945f4d9991293b63c2babc50 P 04474360ddcb4b2a8686a22455345733 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:55:04.312028 31400 raft_consensus.cc:3037] T 2449b162945f4d9991293b63c2babc50 P 04474360ddcb4b2a8686a22455345733 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:55:04.313380 31400 raft_consensus.cc:504] T 2449b162945f4d9991293b63c2babc50 P 04474360ddcb4b2a8686a22455345733 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76b321a2e8c3434a8a225f70d4b24567" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38053 } } peers { permanent_uuid: "04474360ddcb4b2a8686a22455345733" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 43235 } } peers { permanent_uuid: "76aa1f06e7874cde9accdecc25c56921" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 46825 } }
I20260811 18:55:04.313745 31400 leader_election.cc:288] T 2449b162945f4d9991293b63c2babc50 P 04474360ddcb4b2a8686a22455345733 [CANDIDATE]: Term 1 election: Requested vote from peers 76b321a2e8c3434a8a225f70d4b24567 (127.27.180.129:38053), 76aa1f06e7874cde9accdecc25c56921 (127.27.180.131:46825)
I20260811 18:55:04.314097 30759 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2449b162945f4d9991293b63c2babc50" candidate_uuid: "04474360ddcb4b2a8686a22455345733" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76b321a2e8c3434a8a225f70d4b24567"
I20260811 18:55:04.314229 30759 raft_consensus.cc:3037] T 2449b162945f4d9991293b63c2babc50 P 76b321a2e8c3434a8a225f70d4b24567 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:55:04.314170 31275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2449b162945f4d9991293b63c2babc50" candidate_uuid: "04474360ddcb4b2a8686a22455345733" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76aa1f06e7874cde9accdecc25c56921"
I20260811 18:55:04.314317 31275 raft_consensus.cc:3037] T 2449b162945f4d9991293b63c2babc50 P 76aa1f06e7874cde9accdecc25c56921 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:55:04.315340 30759 raft_consensus.cc:2454] T 2449b162945f4d9991293b63c2babc50 P 76b321a2e8c3434a8a225f70d4b24567 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 04474360ddcb4b2a8686a22455345733 in term 1.
I20260811 18:55:04.315340 31275 raft_consensus.cc:2454] T 2449b162945f4d9991293b63c2babc50 P 76aa1f06e7874cde9accdecc25c56921 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 04474360ddcb4b2a8686a22455345733 in term 1.
I20260811 18:55:04.315622 30887 leader_election.cc:302] T 2449b162945f4d9991293b63c2babc50 P 04474360ddcb4b2a8686a22455345733 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04474360ddcb4b2a8686a22455345733, 76aa1f06e7874cde9accdecc25c56921; no voters:
I20260811 18:55:04.315815 31400 raft_consensus.cc:2781] T 2449b162945f4d9991293b63c2babc50 P 04474360ddcb4b2a8686a22455345733 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:55:04.315963 31400 raft_consensus.cc:686] T 2449b162945f4d9991293b63c2babc50 P 04474360ddcb4b2a8686a22455345733 [term 1 LEADER]: Becoming Leader. State: Replica: 04474360ddcb4b2a8686a22455345733, State: Running, Role: LEADER
I20260811 18:55:04.316104 31400 consensus_queue.cc:237] T 2449b162945f4d9991293b63c2babc50 P 04474360ddcb4b2a8686a22455345733 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76b321a2e8c3434a8a225f70d4b24567" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38053 } } peers { permanent_uuid: "04474360ddcb4b2a8686a22455345733" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 43235 } } peers { permanent_uuid: "76aa1f06e7874cde9accdecc25c56921" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 46825 } }
I20260811 18:55:04.316704 31453 tablet_replica.cc:439] T 2449b162945f4d9991293b63c2babc50 P 04474360ddcb4b2a8686a22455345733: TxnStatusTablet state changed. Reason: New leader 04474360ddcb4b2a8686a22455345733. Latest consensus state: current_term: 1 leader_uuid: "04474360ddcb4b2a8686a22455345733" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76b321a2e8c3434a8a225f70d4b24567" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38053 } } peers { permanent_uuid: "04474360ddcb4b2a8686a22455345733" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 43235 } } peers { permanent_uuid: "76aa1f06e7874cde9accdecc25c56921" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 46825 } } }
I20260811 18:55:04.316833 31453 tablet_replica.cc:442] T 2449b162945f4d9991293b63c2babc50 P 04474360ddcb4b2a8686a22455345733: This TxnStatusTablet replica's current role is: LEADER
I20260811 18:55:04.317154 31457 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 18:55:04.317364 31457 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 18:55:04.317816 30474 catalog_manager.cc:5495] T 2449b162945f4d9991293b63c2babc50 P 04474360ddcb4b2a8686a22455345733 reported cstate change: term changed from 0 to 1, leader changed from <none> to 04474360ddcb4b2a8686a22455345733 (127.27.180.130). New cstate: current_term: 1 leader_uuid: "04474360ddcb4b2a8686a22455345733" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76b321a2e8c3434a8a225f70d4b24567" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38053 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "04474360ddcb4b2a8686a22455345733" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 43235 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "76aa1f06e7874cde9accdecc25c56921" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 46825 } health_report { overall_health: UNKNOWN } } }
I20260811 18:55:04.366142 31446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:04.362820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.130:43235 (local address 127.0.0.1:37606)
0811 18:55:04.362981 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:04.362998 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:04.363110 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:04.363400 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:04.363403 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:04.363410 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 18:55:04.363512 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:04.363517 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:04.364279 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:04.364283 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:04.364966 (+ 683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:04.364977 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:04.365428 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:04.365432 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:04.365513 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:04.365556 (+ 43us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 18:55:04.365840 (+ 284us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 18:55:04.365856 (+ 16us) client_negotiation.cc:769] Sending connection context
0811 18:55:04.365917 (+ 61us) client_negotiation.cc:240] Negotiation successful
0811 18:55:04.366021 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260811 18:55:04.366142 31380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:04.362907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37606 (local address 127.27.180.130:43235)
0811 18:55:04.363021 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:04.363025 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:04.363041 (+ 16us) server_negotiation.cc:407] Connection header received
0811 18:55:04.363231 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:04.363235 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:04.363284 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:04.363349 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:55:04.363613 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:04.364163 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:04.365104 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:04.365323 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:04.365411 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:04.365687 (+ 276us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:55:04.365759 (+ 72us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:55:04.365826 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:04.365952 (+ 126us) server_negotiation.cc:299] Negotiation successful
0811 18:55:04.366004 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 18:55:04.369289 31275 raft_consensus.cc:1261] T 2449b162945f4d9991293b63c2babc50 P 76aa1f06e7874cde9accdecc25c56921 [term 1 FOLLOWER]: Refusing update from remote peer 04474360ddcb4b2a8686a22455345733: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:55:04.369580 30759 raft_consensus.cc:1261] T 2449b162945f4d9991293b63c2babc50 P 76b321a2e8c3434a8a225f70d4b24567 [term 1 FOLLOWER]: Refusing update from remote peer 04474360ddcb4b2a8686a22455345733: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:55:04.370047 31453 consensus_queue.cc:1035] T 2449b162945f4d9991293b63c2babc50 P 04474360ddcb4b2a8686a22455345733 [LEADER]: Connected to new peer: Peer: permanent_uuid: "76aa1f06e7874cde9accdecc25c56921" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 46825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:55:04.370198 31400 consensus_queue.cc:1035] T 2449b162945f4d9991293b63c2babc50 P 04474360ddcb4b2a8686a22455345733 [LEADER]: Connected to new peer: Peer: permanent_uuid: "76b321a2e8c3434a8a225f70d4b24567" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:55:04.371721 31398 tablet_replica.cc:439] T 2449b162945f4d9991293b63c2babc50 P 76aa1f06e7874cde9accdecc25c56921: TxnStatusTablet state changed. Reason: New leader 04474360ddcb4b2a8686a22455345733. Latest consensus state: current_term: 1 leader_uuid: "04474360ddcb4b2a8686a22455345733" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76b321a2e8c3434a8a225f70d4b24567" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38053 } } peers { permanent_uuid: "04474360ddcb4b2a8686a22455345733" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 43235 } } peers { permanent_uuid: "76aa1f06e7874cde9accdecc25c56921" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 46825 } } }
I20260811 18:55:04.371847 31398 tablet_replica.cc:442] T 2449b162945f4d9991293b63c2babc50 P 76aa1f06e7874cde9accdecc25c56921: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 18:55:04.371881 31401 tablet_replica.cc:439] T 2449b162945f4d9991293b63c2babc50 P 76b321a2e8c3434a8a225f70d4b24567: TxnStatusTablet state changed. Reason: New leader 04474360ddcb4b2a8686a22455345733. Latest consensus state: current_term: 1 leader_uuid: "04474360ddcb4b2a8686a22455345733" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76b321a2e8c3434a8a225f70d4b24567" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38053 } } peers { permanent_uuid: "04474360ddcb4b2a8686a22455345733" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 43235 } } peers { permanent_uuid: "76aa1f06e7874cde9accdecc25c56921" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 46825 } } }
I20260811 18:55:04.371999 31401 tablet_replica.cc:442] T 2449b162945f4d9991293b63c2babc50 P 76b321a2e8c3434a8a225f70d4b24567: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 18:55:04.374770 31453 tablet_replica.cc:439] T 2449b162945f4d9991293b63c2babc50 P 04474360ddcb4b2a8686a22455345733: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "04474360ddcb4b2a8686a22455345733" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76b321a2e8c3434a8a225f70d4b24567" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38053 } } peers { permanent_uuid: "04474360ddcb4b2a8686a22455345733" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 43235 } } peers { permanent_uuid: "76aa1f06e7874cde9accdecc25c56921" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 46825 } } }
I20260811 18:55:04.374886 31453 tablet_replica.cc:442] T 2449b162945f4d9991293b63c2babc50 P 04474360ddcb4b2a8686a22455345733: This TxnStatusTablet replica's current role is: LEADER
I20260811 18:55:04.375006 31453 tablet_replica.cc:439] T 2449b162945f4d9991293b63c2babc50 P 04474360ddcb4b2a8686a22455345733: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "04474360ddcb4b2a8686a22455345733" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76b321a2e8c3434a8a225f70d4b24567" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38053 } } peers { permanent_uuid: "04474360ddcb4b2a8686a22455345733" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 43235 } } peers { permanent_uuid: "76aa1f06e7874cde9accdecc25c56921" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 46825 } } }
I20260811 18:55:04.375085 31453 tablet_replica.cc:442] T 2449b162945f4d9991293b63c2babc50 P 04474360ddcb4b2a8686a22455345733: This TxnStatusTablet replica's current role is: LEADER
I20260811 18:55:04.375280 31401 tablet_replica.cc:439] T 2449b162945f4d9991293b63c2babc50 P 76b321a2e8c3434a8a225f70d4b24567: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "04474360ddcb4b2a8686a22455345733" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76b321a2e8c3434a8a225f70d4b24567" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38053 } } peers { permanent_uuid: "04474360ddcb4b2a8686a22455345733" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 43235 } } peers { permanent_uuid: "76aa1f06e7874cde9accdecc25c56921" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 46825 } } }
I20260811 18:55:04.375383 31401 tablet_replica.cc:442] T 2449b162945f4d9991293b63c2babc50 P 76b321a2e8c3434a8a225f70d4b24567: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 18:55:04.375368 31398 tablet_replica.cc:439] T 2449b162945f4d9991293b63c2babc50 P 76aa1f06e7874cde9accdecc25c56921: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "04474360ddcb4b2a8686a22455345733" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76b321a2e8c3434a8a225f70d4b24567" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38053 } } peers { permanent_uuid: "04474360ddcb4b2a8686a22455345733" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 43235 } } peers { permanent_uuid: "76aa1f06e7874cde9accdecc25c56921" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 46825 } } }
I20260811 18:55:04.375470 31398 tablet_replica.cc:442] T 2449b162945f4d9991293b63c2babc50 P 76aa1f06e7874cde9accdecc25c56921: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 18:55:04.375481 31471 mvcc.cc:205] Tried to move back new op lower bound from 7317399569892212736 to 7317399569680080896. Current Snapshot: MvccSnapshot[applied={T|T < 7317399569892212736}]
I20260811 18:55:04.385881 31489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:04.382173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37810 (local address 127.27.180.129:38053)
0811 18:55:04.382396 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:04.382400 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:04.382413 (+ 13us) server_negotiation.cc:407] Connection header received
0811 18:55:04.382580 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:04.382583 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:04.382638 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:04.382743 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:55:04.382990 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:04.383513 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:04.384366 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:04.384609 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:04.384697 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:04.384995 (+ 298us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:55:04.385475 (+ 480us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:55:04.385582 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:04.385666 (+ 84us) server_negotiation.cc:299] Negotiation successful
0811 18:55:04.385724 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":91,"threads_started":1}
I20260811 18:55:04.396296 31380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:04.393151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.129:40901 (local address 127.27.180.130:43235)
0811 18:55:04.393263 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:04.393267 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:04.393520 (+ 253us) server_negotiation.cc:407] Connection header received
0811 18:55:04.393631 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:04.393634 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:04.393683 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:04.393742 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:55:04.393995 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:04.394473 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:04.395305 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:04.395523 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:04.395609 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:04.395880 (+ 271us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:55:04.395951 (+ 71us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:55:04.396008 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:04.396113 (+ 105us) server_negotiation.cc:299] Negotiation successful
0811 18:55:04.396165 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 18:55:04.396318 31492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:04.393044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.130:43235 (local address 127.27.180.129:40901)
0811 18:55:04.393433 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:04.393444 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:04.393541 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:04.393759 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:04.393762 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:04.393770 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 18:55:04.393895 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:04.393901 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:04.394576 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:04.394579 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:04.395191 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:04.395200 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:04.395624 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:04.395628 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:04.395711 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:04.395753 (+ 42us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 18:55:04.396029 (+ 276us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 18:55:04.396036 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 18:55:04.396092 (+ 56us) client_negotiation.cc:240] Negotiation successful
0811 18:55:04.396167 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":118,"threads_started":1}
I20260811 18:55:04.403178 31275 raft_consensus.cc:1261] T f7aa1cf8518c43689302ccaff8dd2416 P 76aa1f06e7874cde9accdecc25c56921 [term 1 FOLLOWER]: Refusing update from remote peer 76b321a2e8c3434a8a225f70d4b24567: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:55:04.403477 31015 raft_consensus.cc:1261] T f7aa1cf8518c43689302ccaff8dd2416 P 04474360ddcb4b2a8686a22455345733 [term 1 FOLLOWER]: Refusing update from remote peer 76b321a2e8c3434a8a225f70d4b24567: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:55:04.404096 31438 consensus_queue.cc:1035] T f7aa1cf8518c43689302ccaff8dd2416 P 76b321a2e8c3434a8a225f70d4b24567 [LEADER]: Connected to new peer: Peer: permanent_uuid: "04474360ddcb4b2a8686a22455345733" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 43235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:55:04.404345 31401 consensus_queue.cc:1035] T f7aa1cf8518c43689302ccaff8dd2416 P 76b321a2e8c3434a8a225f70d4b24567 [LEADER]: Connected to new peer: Peer: permanent_uuid: "76aa1f06e7874cde9accdecc25c56921" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 46825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:55:04.426321 31489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:04.422791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.130:55433 (local address 127.27.180.129:38053)
0811 18:55:04.423037 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:04.423042 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:04.423113 (+ 71us) server_negotiation.cc:407] Connection header received
0811 18:55:04.423259 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:04.423262 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:04.423306 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:04.423364 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:55:04.423588 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:04.424094 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:04.425048 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:04.425437 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:04.425557 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:04.425832 (+ 275us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:55:04.425939 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:55:04.426011 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:04.426154 (+ 143us) server_negotiation.cc:299] Negotiation successful
0811 18:55:04.426202 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175}
I20260811 18:55:04.426376 31500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:04.422732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.129:38053 (local address 127.27.180.130:55433)
0811 18:55:04.423047 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:04.423062 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:04.423163 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:04.423386 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:04.423389 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:04.423396 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 18:55:04.423494 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:04.423500 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:04.424193 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:04.424197 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:04.424917 (+ 720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:04.424929 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:04.425564 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:04.425568 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:04.425652 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:04.425706 (+ 54us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 18:55:04.426048 (+ 342us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 18:55:04.426057 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 18:55:04.426132 (+ 75us) client_negotiation.cc:240] Negotiation successful
0811 18:55:04.426215 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":118,"threads_started":1}
I20260811 18:55:04.451397 30474 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50778:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:55:04.451570 30474 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50778:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:55:04.454012 30474 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ad3417ccb7ca4db993c61829c1f3bdbe: Sending DeleteTablet for 3 replicas of tablet f7aa1cf8518c43689302ccaff8dd2416
I20260811 18:55:04.454756 31234 tablet_service.cc:1514] Processing DeleteTablet for tablet f7aa1cf8518c43689302ccaff8dd2416 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:55:04 UTC) from {username='kudu', principal='kudu/127.27.180.190@KRBTEST.COM'} at 127.0.0.1:35480
I20260811 18:55:04.455219 30726 tablet_service.cc:1514] Processing DeleteTablet for tablet f7aa1cf8518c43689302ccaff8dd2416 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:55:04 UTC) from {username='kudu', principal='kudu/127.27.180.190@KRBTEST.COM'} at 127.0.0.1:37806
I20260811 18:55:04.455792 31511 tablet_replica.cc:330] T f7aa1cf8518c43689302ccaff8dd2416 P 76b321a2e8c3434a8a225f70d4b24567: stopping tablet replica
I20260811 18:55:04.455832 30974 tablet_service.cc:1514] Processing DeleteTablet for tablet f7aa1cf8518c43689302ccaff8dd2416 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:55:04 UTC) from {username='kudu', principal='kudu/127.27.180.190@KRBTEST.COM'} at 127.0.0.1:37596
I20260811 18:55:04.455935 31511 raft_consensus.cc:2229] T f7aa1cf8518c43689302ccaff8dd2416 P 76b321a2e8c3434a8a225f70d4b24567 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:55:04.456185 31505 tablet_replica.cc:330] T f7aa1cf8518c43689302ccaff8dd2416 P 76aa1f06e7874cde9accdecc25c56921: stopping tablet replica
I20260811 18:55:04.456267 31511 raft_consensus.cc:2258] T f7aa1cf8518c43689302ccaff8dd2416 P 76b321a2e8c3434a8a225f70d4b24567 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:55:04.456316 31505 raft_consensus.cc:2229] T f7aa1cf8518c43689302ccaff8dd2416 P 76aa1f06e7874cde9accdecc25c56921 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:55:04.456440 31505 raft_consensus.cc:2258] T f7aa1cf8518c43689302ccaff8dd2416 P 76aa1f06e7874cde9accdecc25c56921 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:55:04.457021 31511 ts_tablet_manager.cc:1905] T f7aa1cf8518c43689302ccaff8dd2416 P 76b321a2e8c3434a8a225f70d4b24567: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:55:04.457122 31505 ts_tablet_manager.cc:1905] T f7aa1cf8518c43689302ccaff8dd2416 P 76aa1f06e7874cde9accdecc25c56921: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:55:04.459966 31505 ts_tablet_manager.cc:1918] T f7aa1cf8518c43689302ccaff8dd2416 P 76aa1f06e7874cde9accdecc25c56921: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 18:55:04.460048 31505 log.cc:1196] T f7aa1cf8518c43689302ccaff8dd2416 P 76aa1f06e7874cde9accdecc25c56921: Deleting WAL directory at /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/minicluster-data/ts-2/wal/wals/f7aa1cf8518c43689302ccaff8dd2416
I20260811 18:55:04.460059 31511 ts_tablet_manager.cc:1918] T f7aa1cf8518c43689302ccaff8dd2416 P 76b321a2e8c3434a8a225f70d4b24567: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 18:55:04.460132 31511 log.cc:1196] T f7aa1cf8518c43689302ccaff8dd2416 P 76b321a2e8c3434a8a225f70d4b24567: Deleting WAL directory at /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/minicluster-data/ts-0/wal/wals/f7aa1cf8518c43689302ccaff8dd2416
I20260811 18:55:04.460307 31505 ts_tablet_manager.cc:1939] T f7aa1cf8518c43689302ccaff8dd2416 P 76aa1f06e7874cde9accdecc25c56921: Deleting consensus metadata
I20260811 18:55:04.460417 31511 ts_tablet_manager.cc:1939] T f7aa1cf8518c43689302ccaff8dd2416 P 76b321a2e8c3434a8a225f70d4b24567: Deleting consensus metadata
I20260811 18:55:04.461150 30427 catalog_manager.cc:4841] TS 76b321a2e8c3434a8a225f70d4b24567 (127.27.180.129:38053): tablet f7aa1cf8518c43689302ccaff8dd2416 (table test-table [id=f4756b7337db4d1d9ddd5108ddce22cc]) successfully deleted
I20260811 18:55:04.461153 30425 catalog_manager.cc:4841] TS 76aa1f06e7874cde9accdecc25c56921 (127.27.180.131:46825): tablet f7aa1cf8518c43689302ccaff8dd2416 (table test-table [id=f4756b7337db4d1d9ddd5108ddce22cc]) successfully deleted
I20260811 18:55:04.461539 31512 tablet_replica.cc:330] T f7aa1cf8518c43689302ccaff8dd2416 P 04474360ddcb4b2a8686a22455345733: stopping tablet replica
I20260811 18:55:04.461673 31512 raft_consensus.cc:2229] T f7aa1cf8518c43689302ccaff8dd2416 P 04474360ddcb4b2a8686a22455345733 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:55:04.461800 31512 raft_consensus.cc:2258] T f7aa1cf8518c43689302ccaff8dd2416 P 04474360ddcb4b2a8686a22455345733 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:55:04.462605 31512 ts_tablet_manager.cc:1905] T f7aa1cf8518c43689302ccaff8dd2416 P 04474360ddcb4b2a8686a22455345733: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:55:04.465531 31512 ts_tablet_manager.cc:1918] T f7aa1cf8518c43689302ccaff8dd2416 P 04474360ddcb4b2a8686a22455345733: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 18:55:04.465615 31512 log.cc:1196] T f7aa1cf8518c43689302ccaff8dd2416 P 04474360ddcb4b2a8686a22455345733: Deleting WAL directory at /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786474496157106-28370-0/minicluster-data/ts-1/wal/wals/f7aa1cf8518c43689302ccaff8dd2416
I20260811 18:55:04.465850 31512 ts_tablet_manager.cc:1939] T f7aa1cf8518c43689302ccaff8dd2416 P 04474360ddcb4b2a8686a22455345733: Deleting consensus metadata
I20260811 18:55:04.466622 30425 catalog_manager.cc:4841] TS 04474360ddcb4b2a8686a22455345733 (127.27.180.130:43235): tablet f7aa1cf8518c43689302ccaff8dd2416 (table test-table [id=f4756b7337db4d1d9ddd5108ddce22cc]) successfully deleted
Aug 11 18:55:04 dist-test-slave-2j7r krb5kdc[30388](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474503, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.27.180.129@KRBTEST.COM
I20260811 18:55:04.475049 31489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:04.462254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37822 (local address 127.27.180.129:38053)
0811 18:55:04.462378 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:04.462382 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:04.462942 (+ 560us) server_negotiation.cc:407] Connection header received
0811 18:55:04.463133 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:04.463136 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:04.463184 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:04.463235 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:04.464547 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:04.465404 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:04.466754 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:04.467104 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:04.467175 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:04.471023 (+ 3848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:04.471033 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:04.471051 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:04.471081 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:04.473011 (+ 1930us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:04.473539 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:04.473546 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:04.473548 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:04.473647 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:04.473949 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:04.473953 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:04.473954 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:04.474293 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:04.474526 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:04.474727 (+ 201us) server_negotiation.cc:299] Negotiation successful
0811 18:55:04.474866 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
W20260811 18:55:04.475660 30824 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37822
I20260811 18:55:04.486101 30563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:04.477607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50794 (local address 127.27.180.190:34039)
0811 18:55:04.477806 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:04.477811 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:04.478143 (+ 332us) server_negotiation.cc:407] Connection header received
0811 18:55:04.478382 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:04.478385 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:04.478434 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:04.478507 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:04.479329 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:04.479852 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:04.480964 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:04.481272 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:04.481363 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:04.482854 (+ 1491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:04.482857 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:04.482860 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:04.482884 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:04.484160 (+ 1276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:04.484614 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:04.484617 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:04.484619 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:04.484698 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:04.484992 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:04.484995 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:04.484996 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:04.485316 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:04.485423 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:04.485752 (+ 329us) server_negotiation.cc:299] Negotiation successful
0811 18:55:04.485884 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
W20260811 18:55:04.486601 30474 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50794
I20260811 18:55:04.488122 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 30597
I20260811 18:55:04.503012 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 30849
I20260811 18:55:04.519264 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 31121
I20260811 18:55:04.531786 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 30408
2026-08-11T18:55:04Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (3209 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:55:04 dist-test-slave-2j7r krb5kdc[31521](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:55:04 dist-test-slave-2j7r krb5kdc[31521](info): set up 2 sockets
Aug 11 18:55:04 dist-test-slave-2j7r krb5kdc[31521](info): commencing operation
krb5kdc: starting...
W20260811 18:55:05.734721 28370 mini_kdc.cc:121] Time spent starting KDC: real 1.170s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/krb5kdc/test-user.keytab.
Aug 11 18:55:05 dist-test-slave-2j7r krb5kdc[31521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474505, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:55:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:55:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.180.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.180.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.180.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab.
Entry for principal kudu/127.27.180.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab.
I20260811 18:55:05.856235 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.190:40919
--webserver_interface=127.27.180.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab
--principal=kudu/127.27.180.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:35551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.180.190:40919
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:55:05.966182 31543 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:55:05.966429 31543 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:55:05.966486 31543 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:55:05.970438 31543 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:55:05.970489 31543 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 18:55:05.970512 31543 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:55:05.970535 31543 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:55:05.970564 31543 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:55:05.970587 31543 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:55:05.976212 31543 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:35551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.180.190:40919
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab
--principal=kudu/127.27.180.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.190:40919
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.180.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31543
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:55:05.977439 31543 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:55:05.978433 31543 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:55:05.987308 31550 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:05.987314 31554 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:05.987308 31551 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:55:05.987407 31543 server_base.cc:1034] running on GCE node
I20260811 18:55:05.988049 31543 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:55:05.989142 31543 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:55:05.990314 31543 hybrid_clock.cc:648] HybridClock initialized: now 1786474505990287 us; error 38 us; skew 500 ppm
Aug 11 18:55:05 dist-test-slave-2j7r krb5kdc[31521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474505, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:55:05.993078 31543 init.cc:377] Logged in from keytab as kudu/127.27.180.190@KRBTEST.COM (short username kudu)
I20260811 18:55:05.993992 31543 webserver.cc:460] Webserver started at http://127.27.180.190:43119/ using document root <none> and password file <none>
I20260811 18:55:05.994522 31543 fs_manager.cc:359] Metadata directory not provided
I20260811 18:55:05.994587 31543 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:55:05.994781 31543 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:55:05.996806 31543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/minicluster-data/master-0/data/instance:
uuid: "3b87a006e181434094f7d5896eb20643"
format_stamp: "Formatted at 2026-08-11 18:55:05 on dist-test-slave-2j7r"
server_key: "28af6c7ebc29c7112bcb95d95d53f2d3"
server_key_iv: "37bc0076accdca0eba32199226144898"
server_key_version: "encryptionkey@0"
I20260811 18:55:05.997399 31543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/minicluster-data/master-0/wal/instance:
uuid: "3b87a006e181434094f7d5896eb20643"
format_stamp: "Formatted at 2026-08-11 18:55:05 on dist-test-slave-2j7r"
server_key: "28af6c7ebc29c7112bcb95d95d53f2d3"
server_key_iv: "37bc0076accdca0eba32199226144898"
server_key_version: "encryptionkey@0"
I20260811 18:55:06.001109 31543 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:55:06.003490 31566 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:06.004786 31543 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 18:55:06.004918 31543 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/minicluster-data/master-0/wal
uuid: "3b87a006e181434094f7d5896eb20643"
format_stamp: "Formatted at 2026-08-11 18:55:05 on dist-test-slave-2j7r"
server_key: "28af6c7ebc29c7112bcb95d95d53f2d3"
server_key_iv: "37bc0076accdca0eba32199226144898"
server_key_version: "encryptionkey@0"
I20260811 18:55:06.005038 31543 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:55:06.031214 31543 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:55:06.034237 31543 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:55:06.034463 31543 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:55:06.042587 31543 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.190:40919
I20260811 18:55:06.042603 31681 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.190:40919 every 8 connection(s)
I20260811 18:55:06.043505 31543 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
I20260811 18:55:06.046279 31682 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:55:06.051784 31682 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b87a006e181434094f7d5896eb20643: Bootstrap starting.
I20260811 18:55:06.052136 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 31543
I20260811 18:55:06.052249 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/minicluster-data/master-0/wal/instance
I20260811 18:55:06.052523 28370 external_mini_cluster.cc:1428] Setting key 028546549603ed3b01e1bff37779d8f9
I20260811 18:55:06.054538 31682 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3b87a006e181434094f7d5896eb20643: Neither blocks nor log segments found. Creating new log.
I20260811 18:55:06.055272 31682 log.cc:824] T 00000000000000000000000000000000 P 3b87a006e181434094f7d5896eb20643: Log is configured to *not* fsync() on all Append() calls
I20260811 18:55:06.057395 31682 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b87a006e181434094f7d5896eb20643: No bootstrap required, opened a new log
Aug 11 18:55:06 dist-test-slave-2j7r krb5kdc[31521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474505, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM
I20260811 18:55:06.061462 31682 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3b87a006e181434094f7d5896eb20643 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b87a006e181434094f7d5896eb20643" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 40919 } }
I20260811 18:55:06.061678 31682 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3b87a006e181434094f7d5896eb20643 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:55:06.061764 31682 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3b87a006e181434094f7d5896eb20643 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b87a006e181434094f7d5896eb20643, State: Initialized, Role: FOLLOWER
I20260811 18:55:06.062363 31682 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3b87a006e181434094f7d5896eb20643 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b87a006e181434094f7d5896eb20643" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 40919 } }
I20260811 18:55:06.062554 31682 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3b87a006e181434094f7d5896eb20643 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:55:06.062625 31682 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3b87a006e181434094f7d5896eb20643 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:55:06.062744 31682 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3b87a006e181434094f7d5896eb20643 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:55:06.063642 31682 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3b87a006e181434094f7d5896eb20643 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b87a006e181434094f7d5896eb20643" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 40919 } }
I20260811 18:55:06.063973 31682 leader_election.cc:302] T 00000000000000000000000000000000 P 3b87a006e181434094f7d5896eb20643 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3b87a006e181434094f7d5896eb20643; no voters:
I20260811 18:55:06.064266 31682 leader_election.cc:288] T 00000000000000000000000000000000 P 3b87a006e181434094f7d5896eb20643 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:55:06.064373 31694 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3b87a006e181434094f7d5896eb20643 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:55:06.064541 31694 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3b87a006e181434094f7d5896eb20643 [term 1 LEADER]: Becoming Leader. State: Replica: 3b87a006e181434094f7d5896eb20643, State: Running, Role: LEADER
I20260811 18:55:06.064893 31694 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3b87a006e181434094f7d5896eb20643 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b87a006e181434094f7d5896eb20643" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 40919 } }
I20260811 18:55:06.065591 31682 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3b87a006e181434094f7d5896eb20643 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:55:06.066140 31694 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b87a006e181434094f7d5896eb20643 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3b87a006e181434094f7d5896eb20643. Latest consensus state: current_term: 1 leader_uuid: "3b87a006e181434094f7d5896eb20643" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b87a006e181434094f7d5896eb20643" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 40919 } } }
I20260811 18:55:06.066958 31694 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b87a006e181434094f7d5896eb20643 [sys.catalog]: This master's current role is: LEADER
I20260811 18:55:06.066972 31691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:06.053848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36620 (local address 127.27.180.190:40919)
0811 18:55:06.054257 (+ 409us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:06.054273 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:06.054312 (+ 39us) server_negotiation.cc:407] Connection header received
0811 18:55:06.054997 (+ 685us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:06.055016 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:06.055344 (+ 328us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:06.055649 (+ 305us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:06.056402 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:06.057434 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:06.058301 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:06.059030 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:06.059200 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:06.061810 (+ 2610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:06.061817 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:06.061834 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:06.061866 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:06.063611 (+ 1745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:06.064067 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:06.064074 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:06.064078 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:06.064217 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:06.064519 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:06.064523 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:06.064525 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:06.065032 (+ 507us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:06.065256 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:06.065544 (+ 288us) server_negotiation.cc:299] Negotiation successful
0811 18:55:06.065999 (+ 455us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273,"thread_start_us":125,"threads_started":1}
I20260811 18:55:06.066840 31696 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b87a006e181434094f7d5896eb20643 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3b87a006e181434094f7d5896eb20643" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b87a006e181434094f7d5896eb20643" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 40919 } } }
I20260811 18:55:06.067293 31696 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b87a006e181434094f7d5896eb20643 [sys.catalog]: This master's current role is: LEADER
I20260811 18:55:06.068109 31710 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:55:06.070848 31710 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:55:06.075986 31710 catalog_manager.cc:1347] Generated new cluster ID: ac550a14dedb48e89e18bffacc49ae4c
I20260811 18:55:06.076053 31710 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:55:06.092331 31710 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:55:06.093532 31710 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:55:06.101824 31710 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3b87a006e181434094f7d5896eb20643: Generated new TSK 0
I20260811 18:55:06.102550 31710 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.180.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.180.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.180.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.129.keytab.
Entry for principal kudu/127.27.180.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.129.keytab.
I20260811 18:55:06.139657 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.180.129:0
--local_ip_for_outbound_sockets=127.27.180.129
--webserver_interface=127.27.180.129
--webserver_port=0
--tserver_master_addrs=127.27.180.190:40919
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.129.keytab
--principal=kudu/127.27.180.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:35551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:55:06.249684 31732 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:55:06.249922 31732 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:55:06.249965 31732 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:55:06.253876 31732 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:55:06.253925 31732 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:55:06.253963 31732 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 18:55:06.253978 31732 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 18:55:06.254025 31732 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.180.129
I20260811 18:55:06.259109 31732 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:35551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.129.keytab
--principal=kudu/127.27.180.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.180.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.180.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.27.180.190:40919
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31732
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.180.129
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:55:06.260097 31732 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:55:06.261072 31732 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:55:06.270283 31738 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:06.270295 31739 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:55:06.270445 31732 server_base.cc:1034] running on GCE node
W20260811 18:55:06.270677 31742 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:55:06.271055 31732 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:55:06.271678 31732 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:55:06.272858 31732 hybrid_clock.cc:648] HybridClock initialized: now 1786474506272843 us; error 27 us; skew 500 ppm
Aug 11 18:55:06 dist-test-slave-2j7r krb5kdc[31521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474506, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:55:06.275586 31732 init.cc:377] Logged in from keytab as kudu/127.27.180.129@KRBTEST.COM (short username kudu)
I20260811 18:55:06.276379 31732 webserver.cc:460] Webserver started at http://127.27.180.129:43181/ using document root <none> and password file <none>
I20260811 18:55:06.276886 31732 fs_manager.cc:359] Metadata directory not provided
I20260811 18:55:06.276952 31732 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:55:06.277174 31732 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:55:06.279098 31732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/minicluster-data/ts-0/data/instance:
uuid: "43665cdb5ae6441fbc7ffb7abed2cbaa"
format_stamp: "Formatted at 2026-08-11 18:55:06 on dist-test-slave-2j7r"
server_key: "b2fff3b6b475c63d85fef70dd7e3a620"
server_key_iv: "5ad1dda2a04b94c464d9674f51f18592"
server_key_version: "encryptionkey@0"
I20260811 18:55:06.279636 31732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/minicluster-data/ts-0/wal/instance:
uuid: "43665cdb5ae6441fbc7ffb7abed2cbaa"
format_stamp: "Formatted at 2026-08-11 18:55:06 on dist-test-slave-2j7r"
server_key: "b2fff3b6b475c63d85fef70dd7e3a620"
server_key_iv: "5ad1dda2a04b94c464d9674f51f18592"
server_key_version: "encryptionkey@0"
I20260811 18:55:06.283140 31732 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 18:55:06.285421 31758 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:06.286586 31732 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 18:55:06.286693 31732 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/minicluster-data/ts-0/wal
uuid: "43665cdb5ae6441fbc7ffb7abed2cbaa"
format_stamp: "Formatted at 2026-08-11 18:55:06 on dist-test-slave-2j7r"
server_key: "b2fff3b6b475c63d85fef70dd7e3a620"
server_key_iv: "5ad1dda2a04b94c464d9674f51f18592"
server_key_version: "encryptionkey@0"
I20260811 18:55:06.286784 31732 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:55:06.321023 31732 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:55:06.324169 31732 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:55:06.324370 31732 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:55:06.325886 31732 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:55:06.325954 31732 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:06.326012 31732 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:55:06.326037 31732 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 18:55:06 dist-test-slave-2j7r krb5kdc[31521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474506, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.129@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM
I20260811 18:55:06.336539 31732 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.129:41133
I20260811 18:55:06.336712 31985 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.129:41133 every 8 connection(s)
I20260811 18:55:06.337491 31732 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/minicluster-data/ts-0/data/info.pb
I20260811 18:55:06.339365 31691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:06.326777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.129:48521 (local address 127.27.180.190:40919)
0811 18:55:06.326899 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:06.326902 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:06.327910 (+ 1008us) server_negotiation.cc:407] Connection header received
0811 18:55:06.328940 (+ 1030us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:06.328944 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:06.329003 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:06.329061 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:06.330424 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:06.330987 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:06.332316 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:06.332622 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:06.332742 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:06.335248 (+ 2506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:06.335251 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:06.335253 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:06.335278 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:06.337115 (+ 1837us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:06.337625 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:06.337629 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:06.337631 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:06.337733 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:06.338169 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:06.338192 (+ 23us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:06.338195 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:06.338398 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:06.338500 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:06.339030 (+ 530us) server_negotiation.cc:299] Negotiation successful
0811 18:55:06.339169 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 18:55:06.340032 31779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:06.327288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:40919 (local address 127.27.180.129:48521)
0811 18:55:06.327750 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:06.327793 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:06.328689 (+ 896us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:06.329456 (+ 767us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:06.329462 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:06.329770 (+ 308us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:06.330220 (+ 450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:06.330250 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:06.331122 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:06.331126 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:06.332168 (+ 1042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:06.332179 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:06.332788 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:06.332793 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:06.333019 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:06.333610 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:06.333625 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:06.335080 (+ 1455us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:55:06.337231 (+ 2151us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:06.337238 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:06.337251 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:06.337515 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:06.337830 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:06.337835 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:06.337836 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:06.338035 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:06.338518 (+ 483us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:55:06.338525 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:55:06.338774 (+ 249us) client_negotiation.cc:769] Sending connection context
0811 18:55:06.338952 (+ 178us) client_negotiation.cc:240] Negotiation successful
0811 18:55:06.339262 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":134,"threads_started":1}
I20260811 18:55:06.341348 31986 heartbeater.cc:347] Connected to a master server at 127.27.180.190:40919
I20260811 18:55:06.341495 31986 heartbeater.cc:464] Registering TS with master...
I20260811 18:55:06.342046 31986 heartbeater.cc:511] Master 127.27.180.190:40919 requested a full tablet report, sending...
I20260811 18:55:06.343827 31600 ts_manager.cc:194] Registered new tserver with Master: 43665cdb5ae6441fbc7ffb7abed2cbaa (127.27.180.129:41133)
I20260811 18:55:06.345077 31600 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.180.129@KRBTEST.COM'} at 127.27.180.129:48521
I20260811 18:55:06.346024 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 31732
I20260811 18:55:06.346139 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/minicluster-data/ts-0/wal/instance
I20260811 18:55:06.346436 28370 external_mini_cluster.cc:1428] Setting key 98d5d99c9e5fec17afd4dd27fdc98c0a
I20260811 18:55:06.350258 31997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:06.343396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:40919 (local address 127.27.180.129:56767)
0811 18:55:06.343668 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:06.343682 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:06.343786 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:06.344014 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:06.344017 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:06.344218 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:06.344354 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:06.344362 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:06.345197 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:06.345200 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:06.345894 (+ 694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:06.345903 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:06.346477 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:06.346481 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:06.346570 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:06.346992 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:06.347006 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:06.347404 (+ 398us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:55:06.348897 (+ 1493us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:06.348901 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:06.348903 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:06.349156 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:06.349385 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:06.349388 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:06.349389 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:06.349479 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:06.349780 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:55:06.349783 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:55:06.349884 (+ 101us) client_negotiation.cc:769] Sending connection context
0811 18:55:06.349989 (+ 105us) client_negotiation.cc:240] Negotiation successful
0811 18:55:06.350116 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":90,"threads_started":1}
I20260811 18:55:06.350311 31691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:06.343503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.129:56767 (local address 127.27.180.190:40919)
0811 18:55:06.343653 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:06.343656 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:06.343790 (+ 134us) server_negotiation.cc:407] Connection header received
0811 18:55:06.343869 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:06.343872 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:06.343909 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:06.343996 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:06.344471 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:06.345045 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:06.346021 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:06.346350 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:06.346452 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:06.347527 (+ 1075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:06.347531 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:06.347533 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:06.347555 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:06.348788 (+ 1233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:06.349239 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:06.349242 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:06.349243 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:06.349308 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:06.349551 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:06.349553 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:06.349554 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:06.349691 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:06.349772 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:06.350057 (+ 285us) server_negotiation.cc:299] Negotiation successful
0811 18:55:06.350163 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":79,"server-negotiator.queue_time_us":86}
WARNING: no policy specified for kudu/127.27.180.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.180.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.180.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.130.keytab.
Entry for principal kudu/127.27.180.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.130.keytab.
I20260811 18:55:06.374040 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.180.130:0
--local_ip_for_outbound_sockets=127.27.180.130
--webserver_interface=127.27.180.130
--webserver_port=0
--tserver_master_addrs=127.27.180.190:40919
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.130.keytab
--principal=kudu/127.27.180.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:35551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:55:06.487313 32002 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:55:06.487584 32002 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:55:06.487632 32002 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:55:06.491663 32002 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:55:06.491717 32002 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:55:06.491755 32002 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 18:55:06.491773 32002 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 18:55:06.491803 32002 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.180.130
I20260811 18:55:06.496932 32002 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:35551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.130.keytab
--principal=kudu/127.27.180.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.180.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.180.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.27.180.190:40919
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32002
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.180.130
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:55:06.497912 32002 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:55:06.498852 32002 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:55:06.508564 32015 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:55:06.508627 32002 server_base.cc:1034] running on GCE node
W20260811 18:55:06.508657 32018 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:06.508571 32016 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:55:06.509224 32002 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:55:06.509909 32002 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:55:06.511101 32002 hybrid_clock.cc:648] HybridClock initialized: now 1786474506511082 us; error 31 us; skew 500 ppm
Aug 11 18:55:06 dist-test-slave-2j7r krb5kdc[31521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474506, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:55:06.513911 32002 init.cc:377] Logged in from keytab as kudu/127.27.180.130@KRBTEST.COM (short username kudu)
I20260811 18:55:06.514765 32002 webserver.cc:460] Webserver started at http://127.27.180.130:39871/ using document root <none> and password file <none>
I20260811 18:55:06.515292 32002 fs_manager.cc:359] Metadata directory not provided
I20260811 18:55:06.515360 32002 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:55:06.515560 32002 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:55:06.517668 32002 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/minicluster-data/ts-1/data/instance:
uuid: "ae1ad05a2108484483d65a2d91426f33"
format_stamp: "Formatted at 2026-08-11 18:55:06 on dist-test-slave-2j7r"
server_key: "76ffc4571538c842e9abe3eec2f30962"
server_key_iv: "0890d088e83da5ab2d5d7546c68f470a"
server_key_version: "encryptionkey@0"
I20260811 18:55:06.518193 32002 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/minicluster-data/ts-1/wal/instance:
uuid: "ae1ad05a2108484483d65a2d91426f33"
format_stamp: "Formatted at 2026-08-11 18:55:06 on dist-test-slave-2j7r"
server_key: "76ffc4571538c842e9abe3eec2f30962"
server_key_iv: "0890d088e83da5ab2d5d7546c68f470a"
server_key_version: "encryptionkey@0"
I20260811 18:55:06.522043 32002 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 18:55:06.524575 32026 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:06.525810 32002 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 18:55:06.525956 32002 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/minicluster-data/ts-1/wal
uuid: "ae1ad05a2108484483d65a2d91426f33"
format_stamp: "Formatted at 2026-08-11 18:55:06 on dist-test-slave-2j7r"
server_key: "76ffc4571538c842e9abe3eec2f30962"
server_key_iv: "0890d088e83da5ab2d5d7546c68f470a"
server_key_version: "encryptionkey@0"
I20260811 18:55:06.526063 32002 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:55:06.540802 32002 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:55:06.543637 32002 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:55:06.543809 32002 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:55:06.545272 32002 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:55:06.545329 32002 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:06.545384 32002 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:55:06.545411 32002 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 18:55:06 dist-test-slave-2j7r krb5kdc[31521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474506, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.130@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM
I20260811 18:55:06.555394 32002 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.130:33351
I20260811 18:55:06.555483 32262 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.130:33351 every 8 connection(s)
I20260811 18:55:06.556263 32002 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/minicluster-data/ts-1/data/info.pb
I20260811 18:55:06.558737 31691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:06.546082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.130:59225 (local address 127.27.180.190:40919)
0811 18:55:06.546243 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:06.546249 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:06.547037 (+ 788us) server_negotiation.cc:407] Connection header received
0811 18:55:06.547991 (+ 954us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:06.547994 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:06.548052 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:06.548106 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:06.549775 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:06.550541 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:06.552250 (+ 1709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:06.552571 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:06.552656 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:06.555401 (+ 2745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:06.555404 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:06.555407 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:06.555433 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:06.556841 (+ 1408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:06.557464 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:06.557468 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:06.557469 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:06.557557 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:06.557844 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:06.557846 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:06.557847 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:06.558014 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:06.558088 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:06.558516 (+ 428us) server_negotiation.cc:299] Negotiation successful
0811 18:55:06.558615 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 18:55:06.559437 32048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:06.546422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:40919 (local address 127.27.180.130:59225)
0811 18:55:06.546868 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:06.546918 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:06.547750 (+ 832us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:06.548512 (+ 762us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:06.548522 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:06.548974 (+ 452us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:06.549624 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:06.549645 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:06.550691 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:06.550695 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:06.552111 (+ 1416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:06.552125 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:06.552742 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:06.552747 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:06.552939 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:06.553534 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:06.553555 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:06.555227 (+ 1672us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:55:06.556968 (+ 1741us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:06.556974 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:06.556987 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:06.557335 (+ 348us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:06.557640 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:06.557642 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:06.557643 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:06.557767 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:06.558096 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:55:06.558100 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:55:06.558325 (+ 225us) client_negotiation.cc:769] Sending connection context
0811 18:55:06.558465 (+ 140us) client_negotiation.cc:240] Negotiation successful
0811 18:55:06.558690 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":110,"threads_started":1}
I20260811 18:55:06.560248 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 32002
I20260811 18:55:06.560364 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/minicluster-data/ts-1/wal/instance
I20260811 18:55:06.560668 28370 external_mini_cluster.cc:1428] Setting key 5cd5ee7d3f12e268c381c9c4e8d92348
I20260811 18:55:06.560731 32263 heartbeater.cc:347] Connected to a master server at 127.27.180.190:40919
I20260811 18:55:06.560885 32263 heartbeater.cc:464] Registering TS with master...
I20260811 18:55:06.561448 32263 heartbeater.cc:511] Master 127.27.180.190:40919 requested a full tablet report, sending...
I20260811 18:55:06.562562 31600 ts_manager.cc:194] Registered new tserver with Master: ae1ad05a2108484483d65a2d91426f33 (127.27.180.130:33351)
I20260811 18:55:06.563266 31600 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.180.130@KRBTEST.COM'} at 127.27.180.130:59225
I20260811 18:55:06.571587 32274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:06.562863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:40919 (local address 127.27.180.130:39615)
0811 18:55:06.563147 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:06.563162 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:06.563252 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:06.564971 (+ 1719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:06.564973 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:06.565197 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:06.565331 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:06.565338 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:06.566288 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:06.566291 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:06.566933 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:06.566943 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:06.567505 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:06.567509 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:06.567688 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:06.568128 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:06.568143 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:06.568460 (+ 317us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:55:06.570182 (+ 1722us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:06.570184 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:06.570186 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:06.570393 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:06.570658 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:06.570660 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:06.570661 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:06.570728 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:06.571174 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:55:06.571176 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:55:06.571244 (+ 68us) client_negotiation.cc:769] Sending connection context
0811 18:55:06.571334 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 18:55:06.571436 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":100,"threads_started":1}
I20260811 18:55:06.571681 31691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:06.563533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.130:39615 (local address 127.27.180.190:40919)
0811 18:55:06.563646 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:06.563649 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:06.563661 (+ 12us) server_negotiation.cc:407] Connection header received
0811 18:55:06.563689 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:06.563690 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:06.563721 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:06.563787 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:06.565445 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:06.566136 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:06.567059 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:06.567386 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:06.567491 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:06.568599 (+ 1108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:06.568603 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:06.568606 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:06.568633 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:06.570077 (+ 1444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:06.570489 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:06.570492 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:06.570494 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:06.570575 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:06.570805 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:06.570808 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:06.570810 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:06.571086 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:06.571198 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:06.571418 (+ 220us) server_negotiation.cc:299] Negotiation successful
0811 18:55:06.571531 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
WARNING: no policy specified for kudu/127.27.180.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.180.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.180.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.131.keytab.
Entry for principal kudu/127.27.180.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.131.keytab.
I20260811 18:55:06.589862 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.180.131:0
--local_ip_for_outbound_sockets=127.27.180.131
--webserver_interface=127.27.180.131
--webserver_port=0
--tserver_master_addrs=127.27.180.190:40919
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.131.keytab
--principal=kudu/127.27.180.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:35551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:55:06.697537 32278 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:55:06.697780 32278 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:55:06.697844 32278 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:55:06.701779 32278 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:55:06.701834 32278 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:55:06.701889 32278 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 18:55:06.701915 32278 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 18:55:06.701962 32278 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.180.131
I20260811 18:55:06.707113 32278 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:35551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.131.keytab
--principal=kudu/127.27.180.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.180.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.180.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.27.180.190:40919
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32278
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.180.131
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:55:06.708074 32278 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:55:06.709057 32278 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:55:06.718690 32290 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:06.718806 32287 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:06.718653 32293 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:55:06.718856 32278 server_base.cc:1034] running on GCE node
I20260811 18:55:06.719372 32278 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:55:06.719971 32278 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:55:06.721160 32278 hybrid_clock.cc:648] HybridClock initialized: now 1786474506721141 us; error 32 us; skew 500 ppm
Aug 11 18:55:06 dist-test-slave-2j7r krb5kdc[31521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474506, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:55:06.723836 32278 init.cc:377] Logged in from keytab as kudu/127.27.180.131@KRBTEST.COM (short username kudu)
I20260811 18:55:06.724717 32278 webserver.cc:460] Webserver started at http://127.27.180.131:33975/ using document root <none> and password file <none>
I20260811 18:55:06.725296 32278 fs_manager.cc:359] Metadata directory not provided
I20260811 18:55:06.725365 32278 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:55:06.725569 32278 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:55:06.727496 32278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/minicluster-data/ts-2/data/instance:
uuid: "55f3851f9ad643c49db10785cd71553a"
format_stamp: "Formatted at 2026-08-11 18:55:06 on dist-test-slave-2j7r"
server_key: "172201b7eee29b369436cc6eb3d1eea3"
server_key_iv: "9258d6896c73bd409bd6f804a44083cd"
server_key_version: "encryptionkey@0"
I20260811 18:55:06.728075 32278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/minicluster-data/ts-2/wal/instance:
uuid: "55f3851f9ad643c49db10785cd71553a"
format_stamp: "Formatted at 2026-08-11 18:55:06 on dist-test-slave-2j7r"
server_key: "172201b7eee29b369436cc6eb3d1eea3"
server_key_iv: "9258d6896c73bd409bd6f804a44083cd"
server_key_version: "encryptionkey@0"
I20260811 18:55:06.731774 32278 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260811 18:55:06.734021 32303 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:06.735127 32278 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 18:55:06.735241 32278 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/minicluster-data/ts-2/wal
uuid: "55f3851f9ad643c49db10785cd71553a"
format_stamp: "Formatted at 2026-08-11 18:55:06 on dist-test-slave-2j7r"
server_key: "172201b7eee29b369436cc6eb3d1eea3"
server_key_iv: "9258d6896c73bd409bd6f804a44083cd"
server_key_version: "encryptionkey@0"
I20260811 18:55:06.735349 32278 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:55:06.754621 32278 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:55:06.757360 32278 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:55:06.757531 32278 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:55:06.758852 32278 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:55:06.758900 32278 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:06.758950 32278 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:55:06.758980 32278 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 18:55:06 dist-test-slave-2j7r krb5kdc[31521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474506, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.131@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM
I20260811 18:55:06.769202 32278 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.131:35085
I20260811 18:55:06.769250 32504 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.131:35085 every 8 connection(s)
I20260811 18:55:06.770098 32278 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/minicluster-data/ts-2/data/info.pb
I20260811 18:55:06.772009 31691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:06.759582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.131:35033 (local address 127.27.180.190:40919)
0811 18:55:06.759739 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:06.759744 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:06.760521 (+ 777us) server_negotiation.cc:407] Connection header received
0811 18:55:06.761684 (+ 1163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:06.761688 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:06.761747 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:06.761817 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:06.763116 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:06.763911 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:06.765246 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:06.765550 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:06.765656 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:06.768547 (+ 2891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:06.768550 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:06.768552 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:06.768577 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:06.770008 (+ 1431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:06.770564 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:06.770568 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:06.770570 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:06.770667 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:06.770984 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:06.770988 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:06.770989 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:06.771218 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:06.771324 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:06.771776 (+ 452us) server_negotiation.cc:299] Negotiation successful
0811 18:55:06.771890 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 18:55:06.772692 32315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:06.759903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:40919 (local address 127.27.180.131:35033)
0811 18:55:06.760343 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:06.760415 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:06.761439 (+ 1024us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:06.762158 (+ 719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:06.762164 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:06.762470 (+ 306us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:06.762964 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:06.762983 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:06.764075 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:06.764079 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:06.765134 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:06.765145 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:06.765711 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:06.765715 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:06.765949 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:06.766538 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:06.766555 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:06.768373 (+ 1818us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:55:06.770127 (+ 1754us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:06.770133 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:06.770145 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:06.770460 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:06.770752 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:06.770755 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:06.770757 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:06.770890 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:06.771324 (+ 434us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:55:06.771328 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:55:06.771569 (+ 241us) client_negotiation.cc:769] Sending connection context
0811 18:55:06.771715 (+ 146us) client_negotiation.cc:240] Negotiation successful
0811 18:55:06.771945 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":110,"threads_started":1}
I20260811 18:55:06.773897 32506 heartbeater.cc:347] Connected to a master server at 127.27.180.190:40919
I20260811 18:55:06.774034 32506 heartbeater.cc:464] Registering TS with master...
I20260811 18:55:06.774487 32506 heartbeater.cc:511] Master 127.27.180.190:40919 requested a full tablet report, sending...
I20260811 18:55:06.775621 31592 ts_manager.cc:194] Registered new tserver with Master: 55f3851f9ad643c49db10785cd71553a (127.27.180.131:35085)
I20260811 18:55:06.775722 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 32278
I20260811 18:55:06.775827 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/minicluster-data/ts-2/wal/instance
I20260811 18:55:06.776149 28370 external_mini_cluster.cc:1428] Setting key 3d082b9dc4c8b11cbe1ce64499fbc489
I20260811 18:55:06.776378 31592 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.180.131@KRBTEST.COM'} at 127.27.180.131:35033
I20260811 18:55:06.778316 28370 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 18:55:06.783002 31691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:06.776171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.131:43711 (local address 127.27.180.190:40919)
0811 18:55:06.776303 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:06.776307 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:06.776702 (+ 395us) server_negotiation.cc:407] Connection header received
0811 18:55:06.776854 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:06.776857 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:06.776899 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:06.776968 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:06.777450 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:06.778164 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:06.779063 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:06.779325 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:06.779426 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:06.780349 (+ 923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:06.780353 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:06.780355 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:06.780381 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:06.781632 (+ 1251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:06.782042 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:06.782044 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:06.782046 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:06.782109 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:06.782344 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:06.782346 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:06.782347 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:06.782487 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:06.782556 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:06.782815 (+ 259us) server_negotiation.cc:299] Negotiation successful
0811 18:55:06.782899 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 18:55:06.783072 32514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:06.776088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:40919 (local address 127.27.180.131:43711)
0811 18:55:06.776629 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:06.776643 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:06.776764 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:06.777010 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:06.777013 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:06.777223 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:06.777354 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:06.777363 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:06.778298 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:06.778302 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:06.778943 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:06.778952 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:06.779430 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:06.779432 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:06.779498 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:06.779906 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:06.779918 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:06.780212 (+ 294us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:55:06.781746 (+ 1534us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:06.781750 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:06.781752 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:06.781958 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:06.782179 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:06.782181 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:06.782183 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:06.782272 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:06.782566 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:55:06.782569 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:55:06.782655 (+ 86us) client_negotiation.cc:769] Sending connection context
0811 18:55:06.782760 (+ 105us) client_negotiation.cc:240] Negotiation successful
0811 18:55:06.782881 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":81,"threads_started":1}
Aug 11 18:55:06 dist-test-slave-2j7r krb5kdc[31521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474506, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 18:55:06 dist-test-slave-2j7r krb5kdc[31521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474506, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM
I20260811 18:55:06.803648 31691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:06.796599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42148 (local address 127.27.180.190:40919)
0811 18:55:06.796735 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:06.796739 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:06.796751 (+ 12us) server_negotiation.cc:407] Connection header received
0811 18:55:06.796785 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:06.796788 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:06.796821 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:06.796889 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:06.797550 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:06.798057 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:06.798864 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:06.799067 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:06.799136 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:06.801280 (+ 2144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:06.801283 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:06.801285 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:06.801308 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:06.802442 (+ 1134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:06.802787 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:06.802789 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:06.802790 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:06.802855 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:06.803039 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:06.803040 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:06.803041 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:06.803188 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:06.803278 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:06.803442 (+ 164us) server_negotiation.cc:299] Negotiation successful
0811 18:55:06.803530 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 18:55:06.806327 31592 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42148:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:55:06.809386 31592 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:55:06.822773 32534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:06.817255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48190 (local address 127.27.180.131:35085)
0811 18:55:06.818258 (+ 1003us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:06.818272 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:06.818292 (+ 20us) server_negotiation.cc:407] Connection header received
0811 18:55:06.818352 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:06.818356 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:06.818504 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:06.818637 (+ 133us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:55:06.818907 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:06.819838 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:06.821323 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:06.822120 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:06.822246 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:06.822396 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:06.822538 (+ 142us) server_negotiation.cc:299] Negotiation successful
0811 18:55:06.822628 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":903,"thread_start_us":129,"threads_started":1}
I20260811 18:55:06.822758 32533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:06.817037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.131:35085 (local address 127.0.0.1:48190)
0811 18:55:06.817499 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:06.817549 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:06.817668 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:06.818652 (+ 984us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:06.818658 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:06.818678 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:55:06.818796 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:06.818802 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:06.819955 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:06.819958 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:06.821180 (+ 1222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:06.821191 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:06.822230 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:06.822233 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:06.822351 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:06.822390 (+ 39us) client_negotiation.cc:769] Sending connection context
0811 18:55:06.822497 (+ 107us) client_negotiation.cc:240] Negotiation successful
0811 18:55:06.822620 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":119,"threads_started":1}
I20260811 18:55:06.822979 32535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:06.817410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.129:41133 (local address 127.0.0.1:51424)
0811 18:55:06.817919 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:06.817932 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:06.818016 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:06.818777 (+ 761us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:06.818780 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:06.818812 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:55:06.818918 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:06.818924 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:06.820379 (+ 1455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:06.820382 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:06.821551 (+ 1169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:06.821560 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:06.822591 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:06.822595 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:06.822695 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:06.822720 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 18:55:06.822782 (+ 62us) client_negotiation.cc:240] Negotiation successful
0811 18:55:06.822850 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":98,"threads_started":1}
I20260811 18:55:06.823563 32542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:06.817789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.130:33351 (local address 127.0.0.1:52818)
0811 18:55:06.817998 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:06.818016 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:06.818095 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:06.819679 (+ 1584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:06.819683 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:06.819702 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:55:06.819801 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:06.819806 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:06.820907 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:06.820911 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:06.821992 (+ 1081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:06.822004 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:06.823189 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:06.823192 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:06.823298 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:06.823319 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 18:55:06.823380 (+ 61us) client_negotiation.cc:240] Negotiation successful
0811 18:55:06.823443 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":94,"threads_started":1}
I20260811 18:55:06.823586 32543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:06.818023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51424 (local address 127.27.180.129:41133)
0811 18:55:06.818287 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:06.818304 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:06.818328 (+ 24us) server_negotiation.cc:407] Connection header received
0811 18:55:06.818401 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:06.818406 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:06.818611 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:06.819068 (+ 457us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:55:06.819203 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:06.820253 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:06.821681 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:06.822455 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:06.823200 (+ 745us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:06.823328 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:06.823375 (+ 47us) server_negotiation.cc:299] Negotiation successful
0811 18:55:06.823456 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":93,"threads_started":1}
I20260811 18:55:06.824105 32545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:06.818412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52818 (local address 127.27.180.130:33351)
0811 18:55:06.819249 (+ 837us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:06.819262 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:06.819283 (+ 21us) server_negotiation.cc:407] Connection header received
0811 18:55:06.819364 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:06.819369 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:06.819537 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:06.819679 (+ 142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:55:06.819915 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:06.820792 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:06.822130 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:06.822921 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:06.823685 (+ 764us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:06.823833 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:06.823881 (+ 48us) server_negotiation.cc:299] Negotiation successful
0811 18:55:06.823959 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":733,"thread_start_us":108,"threads_started":1}
I20260811 18:55:06.826050 31862 tablet_service.cc:1467] Processing CreateTablet for tablet d7ff45b6bf794be2b5f0beb8e619f54c (DEFAULT_TABLE table=test-table [id=7adb8e566f3546c8b29f671e7d142952]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:55:06.826064 32400 tablet_service.cc:1467] Processing CreateTablet for tablet d7ff45b6bf794be2b5f0beb8e619f54c (DEFAULT_TABLE table=test-table [id=7adb8e566f3546c8b29f671e7d142952]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:55:06.826982 31862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d7ff45b6bf794be2b5f0beb8e619f54c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:55:06.826624 32136 tablet_service.cc:1467] Processing CreateTablet for tablet d7ff45b6bf794be2b5f0beb8e619f54c (DEFAULT_TABLE table=test-table [id=7adb8e566f3546c8b29f671e7d142952]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:55:06.827111 32400 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d7ff45b6bf794be2b5f0beb8e619f54c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:55:06.827505 32136 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d7ff45b6bf794be2b5f0beb8e619f54c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:55:06.834175 32546 tablet_bootstrap.cc:492] T d7ff45b6bf794be2b5f0beb8e619f54c P 43665cdb5ae6441fbc7ffb7abed2cbaa: Bootstrap starting.
I20260811 18:55:06.834688 32548 tablet_bootstrap.cc:492] T d7ff45b6bf794be2b5f0beb8e619f54c P ae1ad05a2108484483d65a2d91426f33: Bootstrap starting.
I20260811 18:55:06.835093 32551 tablet_bootstrap.cc:492] T d7ff45b6bf794be2b5f0beb8e619f54c P 55f3851f9ad643c49db10785cd71553a: Bootstrap starting.
I20260811 18:55:06.836432 32546 tablet_bootstrap.cc:654] T d7ff45b6bf794be2b5f0beb8e619f54c P 43665cdb5ae6441fbc7ffb7abed2cbaa: Neither blocks nor log segments found. Creating new log.
I20260811 18:55:06.837212 32551 tablet_bootstrap.cc:654] T d7ff45b6bf794be2b5f0beb8e619f54c P 55f3851f9ad643c49db10785cd71553a: Neither blocks nor log segments found. Creating new log.
I20260811 18:55:06.837330 32546 log.cc:824] T d7ff45b6bf794be2b5f0beb8e619f54c P 43665cdb5ae6441fbc7ffb7abed2cbaa: Log is configured to *not* fsync() on all Append() calls
I20260811 18:55:06.837988 32551 log.cc:824] T d7ff45b6bf794be2b5f0beb8e619f54c P 55f3851f9ad643c49db10785cd71553a: Log is configured to *not* fsync() on all Append() calls
I20260811 18:55:06.839112 32548 tablet_bootstrap.cc:654] T d7ff45b6bf794be2b5f0beb8e619f54c P ae1ad05a2108484483d65a2d91426f33: Neither blocks nor log segments found. Creating new log.
I20260811 18:55:06.839537 32546 tablet_bootstrap.cc:492] T d7ff45b6bf794be2b5f0beb8e619f54c P 43665cdb5ae6441fbc7ffb7abed2cbaa: No bootstrap required, opened a new log
I20260811 18:55:06.839789 32546 ts_tablet_manager.cc:1397] T d7ff45b6bf794be2b5f0beb8e619f54c P 43665cdb5ae6441fbc7ffb7abed2cbaa: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.005s
I20260811 18:55:06.840010 32551 tablet_bootstrap.cc:492] T d7ff45b6bf794be2b5f0beb8e619f54c P 55f3851f9ad643c49db10785cd71553a: No bootstrap required, opened a new log
I20260811 18:55:06.839977 32548 log.cc:824] T d7ff45b6bf794be2b5f0beb8e619f54c P ae1ad05a2108484483d65a2d91426f33: Log is configured to *not* fsync() on all Append() calls
I20260811 18:55:06.840179 32551 ts_tablet_manager.cc:1397] T d7ff45b6bf794be2b5f0beb8e619f54c P 55f3851f9ad643c49db10785cd71553a: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260811 18:55:06.842171 32548 tablet_bootstrap.cc:492] T d7ff45b6bf794be2b5f0beb8e619f54c P ae1ad05a2108484483d65a2d91426f33: No bootstrap required, opened a new log
I20260811 18:55:06.842345 32548 ts_tablet_manager.cc:1397] T d7ff45b6bf794be2b5f0beb8e619f54c P ae1ad05a2108484483d65a2d91426f33: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260811 18:55:06.843304 32551 raft_consensus.cc:348] T d7ff45b6bf794be2b5f0beb8e619f54c P 55f3851f9ad643c49db10785cd71553a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55f3851f9ad643c49db10785cd71553a" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 35085 } } peers { permanent_uuid: "ae1ad05a2108484483d65a2d91426f33" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 33351 } } peers { permanent_uuid: "43665cdb5ae6441fbc7ffb7abed2cbaa" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 41133 } }
I20260811 18:55:06.843504 32551 raft_consensus.cc:374] T d7ff45b6bf794be2b5f0beb8e619f54c P 55f3851f9ad643c49db10785cd71553a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:55:06.843408 32546 raft_consensus.cc:348] T d7ff45b6bf794be2b5f0beb8e619f54c P 43665cdb5ae6441fbc7ffb7abed2cbaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55f3851f9ad643c49db10785cd71553a" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 35085 } } peers { permanent_uuid: "ae1ad05a2108484483d65a2d91426f33" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 33351 } } peers { permanent_uuid: "43665cdb5ae6441fbc7ffb7abed2cbaa" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 41133 } }
I20260811 18:55:06.843585 32551 raft_consensus.cc:729] T d7ff45b6bf794be2b5f0beb8e619f54c P 55f3851f9ad643c49db10785cd71553a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55f3851f9ad643c49db10785cd71553a, State: Initialized, Role: FOLLOWER
I20260811 18:55:06.843655 32546 raft_consensus.cc:374] T d7ff45b6bf794be2b5f0beb8e619f54c P 43665cdb5ae6441fbc7ffb7abed2cbaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:55:06.843716 32546 raft_consensus.cc:729] T d7ff45b6bf794be2b5f0beb8e619f54c P 43665cdb5ae6441fbc7ffb7abed2cbaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43665cdb5ae6441fbc7ffb7abed2cbaa, State: Initialized, Role: FOLLOWER
I20260811 18:55:06.844044 32551 consensus_queue.cc:260] T d7ff45b6bf794be2b5f0beb8e619f54c P 55f3851f9ad643c49db10785cd71553a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55f3851f9ad643c49db10785cd71553a" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 35085 } } peers { permanent_uuid: "ae1ad05a2108484483d65a2d91426f33" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 33351 } } peers { permanent_uuid: "43665cdb5ae6441fbc7ffb7abed2cbaa" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 41133 } }
I20260811 18:55:06.844115 32546 consensus_queue.cc:260] T d7ff45b6bf794be2b5f0beb8e619f54c P 43665cdb5ae6441fbc7ffb7abed2cbaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55f3851f9ad643c49db10785cd71553a" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 35085 } } peers { permanent_uuid: "ae1ad05a2108484483d65a2d91426f33" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 33351 } } peers { permanent_uuid: "43665cdb5ae6441fbc7ffb7abed2cbaa" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 41133 } }
I20260811 18:55:06.844621 32506 heartbeater.cc:503] Master 127.27.180.190:40919 was elected leader, sending a full tablet report...
I20260811 18:55:06.845129 32546 ts_tablet_manager.cc:1428] T d7ff45b6bf794be2b5f0beb8e619f54c P 43665cdb5ae6441fbc7ffb7abed2cbaa: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260811 18:55:06.845456 31986 heartbeater.cc:503] Master 127.27.180.190:40919 was elected leader, sending a full tablet report...
I20260811 18:55:06.846490 32551 ts_tablet_manager.cc:1428] T d7ff45b6bf794be2b5f0beb8e619f54c P 55f3851f9ad643c49db10785cd71553a: Time spent starting tablet: real 0.006s user 0.004s sys 0.001s
I20260811 18:55:06.847749 32548 raft_consensus.cc:348] T d7ff45b6bf794be2b5f0beb8e619f54c P ae1ad05a2108484483d65a2d91426f33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55f3851f9ad643c49db10785cd71553a" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 35085 } } peers { permanent_uuid: "ae1ad05a2108484483d65a2d91426f33" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 33351 } } peers { permanent_uuid: "43665cdb5ae6441fbc7ffb7abed2cbaa" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 41133 } }
I20260811 18:55:06.848007 32548 raft_consensus.cc:374] T d7ff45b6bf794be2b5f0beb8e619f54c P ae1ad05a2108484483d65a2d91426f33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:55:06.848100 32548 raft_consensus.cc:729] T d7ff45b6bf794be2b5f0beb8e619f54c P ae1ad05a2108484483d65a2d91426f33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae1ad05a2108484483d65a2d91426f33, State: Initialized, Role: FOLLOWER
I20260811 18:55:06.848510 32548 consensus_queue.cc:260] T d7ff45b6bf794be2b5f0beb8e619f54c P ae1ad05a2108484483d65a2d91426f33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55f3851f9ad643c49db10785cd71553a" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 35085 } } peers { permanent_uuid: "ae1ad05a2108484483d65a2d91426f33" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 33351 } } peers { permanent_uuid: "43665cdb5ae6441fbc7ffb7abed2cbaa" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 41133 } }
I20260811 18:55:06.849234 32263 heartbeater.cc:503] Master 127.27.180.190:40919 was elected leader, sending a full tablet report...
I20260811 18:55:06.849453 32548 ts_tablet_manager.cc:1428] T d7ff45b6bf794be2b5f0beb8e619f54c P ae1ad05a2108484483d65a2d91426f33: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260811 18:55:06.958113 32560 raft_consensus.cc:482] T d7ff45b6bf794be2b5f0beb8e619f54c P ae1ad05a2108484483d65a2d91426f33 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:55:06.958364 32560 raft_consensus.cc:504] T d7ff45b6bf794be2b5f0beb8e619f54c P ae1ad05a2108484483d65a2d91426f33 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55f3851f9ad643c49db10785cd71553a" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 35085 } } peers { permanent_uuid: "ae1ad05a2108484483d65a2d91426f33" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 33351 } } peers { permanent_uuid: "43665cdb5ae6441fbc7ffb7abed2cbaa" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 41133 } }
I20260811 18:55:06.959522 32560 leader_election.cc:288] T d7ff45b6bf794be2b5f0beb8e619f54c P ae1ad05a2108484483d65a2d91426f33 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 55f3851f9ad643c49db10785cd71553a (127.27.180.131:35085), 43665cdb5ae6441fbc7ffb7abed2cbaa (127.27.180.129:41133)
I20260811 18:55:06.964079 32543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:06.960015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.130:33735 (local address 127.27.180.129:41133)
0811 18:55:06.960123 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:06.960127 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:06.960143 (+ 16us) server_negotiation.cc:407] Connection header received
0811 18:55:06.960280 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:06.960285 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:06.960349 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:06.960448 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:55:06.960776 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:06.961600 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:06.962916 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:06.963493 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:06.963589 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:06.963633 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:06.963895 (+ 262us) server_negotiation.cc:299] Negotiation successful
0811 18:55:06.963956 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 18:55:06.964118 32048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:06.959837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.129:41133 (local address 127.27.180.130:33735)
0811 18:55:06.959992 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:06.960010 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:06.960129 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:06.960476 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:06.960480 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:06.960503 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:55:06.960639 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:06.960647 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:06.961713 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:06.961717 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:06.962776 (+ 1059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:06.962786 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:06.963629 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:06.963633 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:06.963769 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:06.963796 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 18:55:06.963883 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 18:55:06.963959 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260811 18:55:06.964546 32561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:06.959837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.131:35085 (local address 127.27.180.130:51475)
0811 18:55:06.960360 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:06.960374 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:06.960486 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:06.960762 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:06.960764 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:06.960781 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:55:06.961374 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:06.961385 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:06.962368 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:06.962370 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:06.963253 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:06.963264 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:06.964088 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:06.964092 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:06.964189 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:06.964213 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 18:55:06.964329 (+ 116us) client_negotiation.cc:240] Negotiation successful
0811 18:55:06.964400 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":452,"mutex_wait_us":131,"thread_start_us":170,"threads_started":1}
I20260811 18:55:06.964568 32534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:06.959929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.130:51475 (local address 127.27.180.131:35085)
0811 18:55:06.960058 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:06.960063 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:06.960469 (+ 406us) server_negotiation.cc:407] Connection header received
0811 18:55:06.960592 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:06.960595 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:06.960649 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:06.960748 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:55:06.961510 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:06.962243 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:06.963402 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:06.963960 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:06.964070 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:06.964117 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:06.964376 (+ 259us) server_negotiation.cc:299] Negotiation successful
0811 18:55:06.964459 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 18:55:06.964843 31908 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7ff45b6bf794be2b5f0beb8e619f54c" candidate_uuid: "ae1ad05a2108484483d65a2d91426f33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43665cdb5ae6441fbc7ffb7abed2cbaa" is_pre_election: true
I20260811 18:55:06.965207 31908 raft_consensus.cc:2454] T d7ff45b6bf794be2b5f0beb8e619f54c P 43665cdb5ae6441fbc7ffb7abed2cbaa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ae1ad05a2108484483d65a2d91426f33 in term 0.
I20260811 18:55:06.965273 32421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7ff45b6bf794be2b5f0beb8e619f54c" candidate_uuid: "ae1ad05a2108484483d65a2d91426f33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55f3851f9ad643c49db10785cd71553a" is_pre_election: true
I20260811 18:55:06.965547 32421 raft_consensus.cc:2454] T d7ff45b6bf794be2b5f0beb8e619f54c P 55f3851f9ad643c49db10785cd71553a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ae1ad05a2108484483d65a2d91426f33 in term 0.
I20260811 18:55:06.965761 32033 leader_election.cc:302] T d7ff45b6bf794be2b5f0beb8e619f54c P ae1ad05a2108484483d65a2d91426f33 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43665cdb5ae6441fbc7ffb7abed2cbaa, ae1ad05a2108484483d65a2d91426f33; no voters:
I20260811 18:55:06.966137 32560 raft_consensus.cc:2781] T d7ff45b6bf794be2b5f0beb8e619f54c P ae1ad05a2108484483d65a2d91426f33 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:55:06.966230 32560 raft_consensus.cc:482] T d7ff45b6bf794be2b5f0beb8e619f54c P ae1ad05a2108484483d65a2d91426f33 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:55:06.966430 32560 raft_consensus.cc:3037] T d7ff45b6bf794be2b5f0beb8e619f54c P ae1ad05a2108484483d65a2d91426f33 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:55:06.967576 32560 raft_consensus.cc:504] T d7ff45b6bf794be2b5f0beb8e619f54c P ae1ad05a2108484483d65a2d91426f33 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55f3851f9ad643c49db10785cd71553a" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 35085 } } peers { permanent_uuid: "ae1ad05a2108484483d65a2d91426f33" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 33351 } } peers { permanent_uuid: "43665cdb5ae6441fbc7ffb7abed2cbaa" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 41133 } }
I20260811 18:55:06.967895 32560 leader_election.cc:288] T d7ff45b6bf794be2b5f0beb8e619f54c P ae1ad05a2108484483d65a2d91426f33 [CANDIDATE]: Term 1 election: Requested vote from peers 55f3851f9ad643c49db10785cd71553a (127.27.180.131:35085), 43665cdb5ae6441fbc7ffb7abed2cbaa (127.27.180.129:41133)
I20260811 18:55:06.968256 31908 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7ff45b6bf794be2b5f0beb8e619f54c" candidate_uuid: "ae1ad05a2108484483d65a2d91426f33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43665cdb5ae6441fbc7ffb7abed2cbaa"
I20260811 18:55:06.968256 32421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7ff45b6bf794be2b5f0beb8e619f54c" candidate_uuid: "ae1ad05a2108484483d65a2d91426f33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55f3851f9ad643c49db10785cd71553a"
I20260811 18:55:06.968380 31908 raft_consensus.cc:3037] T d7ff45b6bf794be2b5f0beb8e619f54c P 43665cdb5ae6441fbc7ffb7abed2cbaa [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:55:06.968380 32421 raft_consensus.cc:3037] T d7ff45b6bf794be2b5f0beb8e619f54c P 55f3851f9ad643c49db10785cd71553a [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:55:06.969655 31908 raft_consensus.cc:2454] T d7ff45b6bf794be2b5f0beb8e619f54c P 43665cdb5ae6441fbc7ffb7abed2cbaa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae1ad05a2108484483d65a2d91426f33 in term 1.
I20260811 18:55:06.969655 32421 raft_consensus.cc:2454] T d7ff45b6bf794be2b5f0beb8e619f54c P 55f3851f9ad643c49db10785cd71553a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae1ad05a2108484483d65a2d91426f33 in term 1.
I20260811 18:55:06.969966 32037 leader_election.cc:302] T d7ff45b6bf794be2b5f0beb8e619f54c P ae1ad05a2108484483d65a2d91426f33 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55f3851f9ad643c49db10785cd71553a, ae1ad05a2108484483d65a2d91426f33; no voters:
I20260811 18:55:06.970124 32560 raft_consensus.cc:2781] T d7ff45b6bf794be2b5f0beb8e619f54c P ae1ad05a2108484483d65a2d91426f33 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:55:06.970314 32560 raft_consensus.cc:686] T d7ff45b6bf794be2b5f0beb8e619f54c P ae1ad05a2108484483d65a2d91426f33 [term 1 LEADER]: Becoming Leader. State: Replica: ae1ad05a2108484483d65a2d91426f33, State: Running, Role: LEADER
I20260811 18:55:06.970606 32560 consensus_queue.cc:237] T d7ff45b6bf794be2b5f0beb8e619f54c P ae1ad05a2108484483d65a2d91426f33 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55f3851f9ad643c49db10785cd71553a" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 35085 } } peers { permanent_uuid: "ae1ad05a2108484483d65a2d91426f33" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 33351 } } peers { permanent_uuid: "43665cdb5ae6441fbc7ffb7abed2cbaa" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 41133 } }
I20260811 18:55:06.973788 31592 catalog_manager.cc:5495] T d7ff45b6bf794be2b5f0beb8e619f54c P ae1ad05a2108484483d65a2d91426f33 reported cstate change: term changed from 0 to 1, leader changed from <none> to ae1ad05a2108484483d65a2d91426f33 (127.27.180.130). New cstate: current_term: 1 leader_uuid: "ae1ad05a2108484483d65a2d91426f33" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55f3851f9ad643c49db10785cd71553a" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 35085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ae1ad05a2108484483d65a2d91426f33" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 33351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "43665cdb5ae6441fbc7ffb7abed2cbaa" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 41133 } health_report { overall_health: UNKNOWN } } }
W20260811 18:55:07.021886 32507 tablet.cc:2369] T d7ff45b6bf794be2b5f0beb8e619f54c P 55f3851f9ad643c49db10785cd71553a: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 11 18:55:07 dist-test-slave-2j7r krb5kdc[31521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474505, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.190@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM
I20260811 18:55:07.030648 31691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:07.022851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42158 (local address 127.27.180.190:40919)
0811 18:55:07.022996 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:07.023001 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:07.023152 (+ 151us) server_negotiation.cc:407] Connection header received
0811 18:55:07.023272 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:07.023275 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:07.023321 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:07.023386 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:07.024065 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:07.024573 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:07.025393 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:07.025598 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:07.025684 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:07.027779 (+ 2095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:07.027782 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:07.027784 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:07.027807 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:07.028933 (+ 1126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:07.029510 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:07.029513 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:07.029515 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:07.029587 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:07.029832 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:07.029836 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:07.029838 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:07.030003 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:07.030088 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:07.030408 (+ 320us) server_negotiation.cc:299] Negotiation successful
0811 18:55:07.030508 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 18:55:07.030650 32577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:07.022768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:40919 (local address 127.0.0.1:42158)
0811 18:55:07.023063 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:07.023075 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:07.023184 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:07.023428 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:07.023431 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:07.023832 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:07.023936 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:07.023943 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:07.024678 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:07.024681 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:07.025273 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:07.025282 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:07.025699 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:07.025702 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:07.025778 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:07.026287 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:07.026300 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:07.027655 (+ 1355us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:55:07.029031 (+ 1376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:07.029041 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:07.029047 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:07.029406 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:07.029669 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:07.029672 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:07.029674 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:07.029752 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:07.030104 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:55:07.030111 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:55:07.030218 (+ 107us) client_negotiation.cc:769] Sending connection context
0811 18:55:07.030315 (+ 97us) client_negotiation.cc:240] Negotiation successful
0811 18:55:07.030460 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":85,"threads_started":1}
I20260811 18:55:07.034325 31592 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.27.180.190@KRBTEST.COM'} at 127.0.0.1:42158:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 18:55:07.035030 31592 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:55:07.039929 32400 tablet_service.cc:1467] Processing CreateTablet for tablet 3f50532b429d471291e1d41dcf368144 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=cd255858698843d4a2c452929d176466]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 18:55:07.039868 31862 tablet_service.cc:1467] Processing CreateTablet for tablet 3f50532b429d471291e1d41dcf368144 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=cd255858698843d4a2c452929d176466]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 18:55:07.040586 31862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f50532b429d471291e1d41dcf368144. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:55:07.040586 32400 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f50532b429d471291e1d41dcf368144. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:55:07.043072 32546 tablet_bootstrap.cc:492] T 3f50532b429d471291e1d41dcf368144 P 43665cdb5ae6441fbc7ffb7abed2cbaa: Bootstrap starting.
I20260811 18:55:07.043310 32551 tablet_bootstrap.cc:492] T 3f50532b429d471291e1d41dcf368144 P 55f3851f9ad643c49db10785cd71553a: Bootstrap starting.
I20260811 18:55:07.043478 32136 tablet_service.cc:1467] Processing CreateTablet for tablet 3f50532b429d471291e1d41dcf368144 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=cd255858698843d4a2c452929d176466]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 18:55:07.044059 32546 tablet_bootstrap.cc:654] T 3f50532b429d471291e1d41dcf368144 P 43665cdb5ae6441fbc7ffb7abed2cbaa: Neither blocks nor log segments found. Creating new log.
I20260811 18:55:07.044085 32136 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f50532b429d471291e1d41dcf368144. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:55:07.044214 32551 tablet_bootstrap.cc:654] T 3f50532b429d471291e1d41dcf368144 P 55f3851f9ad643c49db10785cd71553a: Neither blocks nor log segments found. Creating new log.
I20260811 18:55:07.045485 32551 tablet_bootstrap.cc:492] T 3f50532b429d471291e1d41dcf368144 P 55f3851f9ad643c49db10785cd71553a: No bootstrap required, opened a new log
I20260811 18:55:07.045571 32551 ts_tablet_manager.cc:1397] T 3f50532b429d471291e1d41dcf368144 P 55f3851f9ad643c49db10785cd71553a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260811 18:55:07.045786 32546 tablet_bootstrap.cc:492] T 3f50532b429d471291e1d41dcf368144 P 43665cdb5ae6441fbc7ffb7abed2cbaa: No bootstrap required, opened a new log
I20260811 18:55:07.045855 32546 ts_tablet_manager.cc:1397] T 3f50532b429d471291e1d41dcf368144 P 43665cdb5ae6441fbc7ffb7abed2cbaa: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 18:55:07.046036 32551 raft_consensus.cc:348] T 3f50532b429d471291e1d41dcf368144 P 55f3851f9ad643c49db10785cd71553a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43665cdb5ae6441fbc7ffb7abed2cbaa" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 41133 } } peers { permanent_uuid: "ae1ad05a2108484483d65a2d91426f33" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 33351 } } peers { permanent_uuid: "55f3851f9ad643c49db10785cd71553a" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 35085 } }
I20260811 18:55:07.046144 32551 raft_consensus.cc:374] T 3f50532b429d471291e1d41dcf368144 P 55f3851f9ad643c49db10785cd71553a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:55:07.046173 32551 raft_consensus.cc:729] T 3f50532b429d471291e1d41dcf368144 P 55f3851f9ad643c49db10785cd71553a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55f3851f9ad643c49db10785cd71553a, State: Initialized, Role: FOLLOWER
I20260811 18:55:07.046284 32546 raft_consensus.cc:348] T 3f50532b429d471291e1d41dcf368144 P 43665cdb5ae6441fbc7ffb7abed2cbaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43665cdb5ae6441fbc7ffb7abed2cbaa" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 41133 } } peers { permanent_uuid: "ae1ad05a2108484483d65a2d91426f33" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 33351 } } peers { permanent_uuid: "55f3851f9ad643c49db10785cd71553a" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 35085 } }
I20260811 18:55:07.046308 32551 consensus_queue.cc:260] T 3f50532b429d471291e1d41dcf368144 P 55f3851f9ad643c49db10785cd71553a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43665cdb5ae6441fbc7ffb7abed2cbaa" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 41133 } } peers { permanent_uuid: "ae1ad05a2108484483d65a2d91426f33" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 33351 } } peers { permanent_uuid: "55f3851f9ad643c49db10785cd71553a" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 35085 } }
I20260811 18:55:07.046379 32546 raft_consensus.cc:374] T 3f50532b429d471291e1d41dcf368144 P 43665cdb5ae6441fbc7ffb7abed2cbaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:55:07.046406 32546 raft_consensus.cc:729] T 3f50532b429d471291e1d41dcf368144 P 43665cdb5ae6441fbc7ffb7abed2cbaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43665cdb5ae6441fbc7ffb7abed2cbaa, State: Initialized, Role: FOLLOWER
I20260811 18:55:07.046512 32546 consensus_queue.cc:260] T 3f50532b429d471291e1d41dcf368144 P 43665cdb5ae6441fbc7ffb7abed2cbaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43665cdb5ae6441fbc7ffb7abed2cbaa" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 41133 } } peers { permanent_uuid: "ae1ad05a2108484483d65a2d91426f33" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 33351 } } peers { permanent_uuid: "55f3851f9ad643c49db10785cd71553a" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 35085 } }
I20260811 18:55:07.046576 32551 ts_tablet_manager.cc:1428] T 3f50532b429d471291e1d41dcf368144 P 55f3851f9ad643c49db10785cd71553a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 18:55:07.046767 32546 ts_tablet_manager.cc:1428] T 3f50532b429d471291e1d41dcf368144 P 43665cdb5ae6441fbc7ffb7abed2cbaa: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 18:55:07.047089 32559 tablet_replica.cc:439] T 3f50532b429d471291e1d41dcf368144 P 43665cdb5ae6441fbc7ffb7abed2cbaa: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43665cdb5ae6441fbc7ffb7abed2cbaa" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 41133 } } peers { permanent_uuid: "ae1ad05a2108484483d65a2d91426f33" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 33351 } } peers { permanent_uuid: "55f3851f9ad643c49db10785cd71553a" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 35085 } } }
I20260811 18:55:07.047158 32548 tablet_bootstrap.cc:492] T 3f50532b429d471291e1d41dcf368144 P ae1ad05a2108484483d65a2d91426f33: Bootstrap starting.
I20260811 18:55:07.047240 32559 tablet_replica.cc:442] T 3f50532b429d471291e1d41dcf368144 P 43665cdb5ae6441fbc7ffb7abed2cbaa: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 18:55:07.046581 32557 tablet_replica.cc:439] T 3f50532b429d471291e1d41dcf368144 P 55f3851f9ad643c49db10785cd71553a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43665cdb5ae6441fbc7ffb7abed2cbaa" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 41133 } } peers { permanent_uuid: "ae1ad05a2108484483d65a2d91426f33" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 33351 } } peers { permanent_uuid: "55f3851f9ad643c49db10785cd71553a" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 35085 } } }
I20260811 18:55:07.047704 32557 tablet_replica.cc:442] T 3f50532b429d471291e1d41dcf368144 P 55f3851f9ad643c49db10785cd71553a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 18:55:07.048020 32548 tablet_bootstrap.cc:654] T 3f50532b429d471291e1d41dcf368144 P ae1ad05a2108484483d65a2d91426f33: Neither blocks nor log segments found. Creating new log.
I20260811 18:55:07.049408 32548 tablet_bootstrap.cc:492] T 3f50532b429d471291e1d41dcf368144 P ae1ad05a2108484483d65a2d91426f33: No bootstrap required, opened a new log
I20260811 18:55:07.049499 32548 ts_tablet_manager.cc:1397] T 3f50532b429d471291e1d41dcf368144 P ae1ad05a2108484483d65a2d91426f33: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:55:07.049891 32548 raft_consensus.cc:348] T 3f50532b429d471291e1d41dcf368144 P ae1ad05a2108484483d65a2d91426f33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43665cdb5ae6441fbc7ffb7abed2cbaa" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 41133 } } peers { permanent_uuid: "ae1ad05a2108484483d65a2d91426f33" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 33351 } } peers { permanent_uuid: "55f3851f9ad643c49db10785cd71553a" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 35085 } }
I20260811 18:55:07.049969 32548 raft_consensus.cc:374] T 3f50532b429d471291e1d41dcf368144 P ae1ad05a2108484483d65a2d91426f33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:55:07.049988 32548 raft_consensus.cc:729] T 3f50532b429d471291e1d41dcf368144 P ae1ad05a2108484483d65a2d91426f33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae1ad05a2108484483d65a2d91426f33, State: Initialized, Role: FOLLOWER
I20260811 18:55:07.050083 32548 consensus_queue.cc:260] T 3f50532b429d471291e1d41dcf368144 P ae1ad05a2108484483d65a2d91426f33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43665cdb5ae6441fbc7ffb7abed2cbaa" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 41133 } } peers { permanent_uuid: "ae1ad05a2108484483d65a2d91426f33" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 33351 } } peers { permanent_uuid: "55f3851f9ad643c49db10785cd71553a" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 35085 } }
I20260811 18:55:07.050350 32548 ts_tablet_manager.cc:1428] T 3f50532b429d471291e1d41dcf368144 P ae1ad05a2108484483d65a2d91426f33: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 18:55:07.050614 32560 tablet_replica.cc:439] T 3f50532b429d471291e1d41dcf368144 P ae1ad05a2108484483d65a2d91426f33: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43665cdb5ae6441fbc7ffb7abed2cbaa" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 41133 } } peers { permanent_uuid: "ae1ad05a2108484483d65a2d91426f33" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 33351 } } peers { permanent_uuid: "55f3851f9ad643c49db10785cd71553a" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 35085 } } }
I20260811 18:55:07.050719 32560 tablet_replica.cc:442] T 3f50532b429d471291e1d41dcf368144 P ae1ad05a2108484483d65a2d91426f33: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 18:55:07.057430 32560 raft_consensus.cc:482] T 3f50532b429d471291e1d41dcf368144 P ae1ad05a2108484483d65a2d91426f33 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260811 18:55:07.057461 32265 tablet.cc:2369] T 3f50532b429d471291e1d41dcf368144 P ae1ad05a2108484483d65a2d91426f33: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:55:07.057528 32560 raft_consensus.cc:504] T 3f50532b429d471291e1d41dcf368144 P ae1ad05a2108484483d65a2d91426f33 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43665cdb5ae6441fbc7ffb7abed2cbaa" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 41133 } } peers { permanent_uuid: "ae1ad05a2108484483d65a2d91426f33" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 33351 } } peers { permanent_uuid: "55f3851f9ad643c49db10785cd71553a" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 35085 } }
I20260811 18:55:07.057868 32560 leader_election.cc:288] T 3f50532b429d471291e1d41dcf368144 P ae1ad05a2108484483d65a2d91426f33 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43665cdb5ae6441fbc7ffb7abed2cbaa (127.27.180.129:41133), 55f3851f9ad643c49db10785cd71553a (127.27.180.131:35085)
I20260811 18:55:07.058233 31908 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f50532b429d471291e1d41dcf368144" candidate_uuid: "ae1ad05a2108484483d65a2d91426f33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43665cdb5ae6441fbc7ffb7abed2cbaa" is_pre_election: true
I20260811 18:55:07.058313 32421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f50532b429d471291e1d41dcf368144" candidate_uuid: "ae1ad05a2108484483d65a2d91426f33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55f3851f9ad643c49db10785cd71553a" is_pre_election: true
I20260811 18:55:07.058394 31908 raft_consensus.cc:2454] T 3f50532b429d471291e1d41dcf368144 P 43665cdb5ae6441fbc7ffb7abed2cbaa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ae1ad05a2108484483d65a2d91426f33 in term 0.
I20260811 18:55:07.058430 32421 raft_consensus.cc:2454] T 3f50532b429d471291e1d41dcf368144 P 55f3851f9ad643c49db10785cd71553a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ae1ad05a2108484483d65a2d91426f33 in term 0.
I20260811 18:55:07.058674 32033 leader_election.cc:302] T 3f50532b429d471291e1d41dcf368144 P ae1ad05a2108484483d65a2d91426f33 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43665cdb5ae6441fbc7ffb7abed2cbaa, ae1ad05a2108484483d65a2d91426f33; no voters:
I20260811 18:55:07.058928 32560 raft_consensus.cc:2781] T 3f50532b429d471291e1d41dcf368144 P ae1ad05a2108484483d65a2d91426f33 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:55:07.059001 32560 raft_consensus.cc:482] T 3f50532b429d471291e1d41dcf368144 P ae1ad05a2108484483d65a2d91426f33 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:55:07.059054 32560 raft_consensus.cc:3037] T 3f50532b429d471291e1d41dcf368144 P ae1ad05a2108484483d65a2d91426f33 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:55:07.059949 32560 raft_consensus.cc:504] T 3f50532b429d471291e1d41dcf368144 P ae1ad05a2108484483d65a2d91426f33 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43665cdb5ae6441fbc7ffb7abed2cbaa" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 41133 } } peers { permanent_uuid: "ae1ad05a2108484483d65a2d91426f33" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 33351 } } peers { permanent_uuid: "55f3851f9ad643c49db10785cd71553a" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 35085 } }
I20260811 18:55:07.060245 32560 leader_election.cc:288] T 3f50532b429d471291e1d41dcf368144 P ae1ad05a2108484483d65a2d91426f33 [CANDIDATE]: Term 1 election: Requested vote from peers 43665cdb5ae6441fbc7ffb7abed2cbaa (127.27.180.129:41133), 55f3851f9ad643c49db10785cd71553a (127.27.180.131:35085)
I20260811 18:55:07.060567 31908 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f50532b429d471291e1d41dcf368144" candidate_uuid: "ae1ad05a2108484483d65a2d91426f33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43665cdb5ae6441fbc7ffb7abed2cbaa"
I20260811 18:55:07.060634 32421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f50532b429d471291e1d41dcf368144" candidate_uuid: "ae1ad05a2108484483d65a2d91426f33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55f3851f9ad643c49db10785cd71553a"
I20260811 18:55:07.060688 31908 raft_consensus.cc:3037] T 3f50532b429d471291e1d41dcf368144 P 43665cdb5ae6441fbc7ffb7abed2cbaa [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:55:07.060741 32421 raft_consensus.cc:3037] T 3f50532b429d471291e1d41dcf368144 P 55f3851f9ad643c49db10785cd71553a [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:55:07.061807 31908 raft_consensus.cc:2454] T 3f50532b429d471291e1d41dcf368144 P 43665cdb5ae6441fbc7ffb7abed2cbaa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae1ad05a2108484483d65a2d91426f33 in term 1.
I20260811 18:55:07.061807 32421 raft_consensus.cc:2454] T 3f50532b429d471291e1d41dcf368144 P 55f3851f9ad643c49db10785cd71553a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae1ad05a2108484483d65a2d91426f33 in term 1.
I20260811 18:55:07.062059 32037 leader_election.cc:302] T 3f50532b429d471291e1d41dcf368144 P ae1ad05a2108484483d65a2d91426f33 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55f3851f9ad643c49db10785cd71553a, ae1ad05a2108484483d65a2d91426f33; no voters:
I20260811 18:55:07.062225 32560 raft_consensus.cc:2781] T 3f50532b429d471291e1d41dcf368144 P ae1ad05a2108484483d65a2d91426f33 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:55:07.062340 32560 raft_consensus.cc:686] T 3f50532b429d471291e1d41dcf368144 P ae1ad05a2108484483d65a2d91426f33 [term 1 LEADER]: Becoming Leader. State: Replica: ae1ad05a2108484483d65a2d91426f33, State: Running, Role: LEADER
I20260811 18:55:07.062479 32560 consensus_queue.cc:237] T 3f50532b429d471291e1d41dcf368144 P ae1ad05a2108484483d65a2d91426f33 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43665cdb5ae6441fbc7ffb7abed2cbaa" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 41133 } } peers { permanent_uuid: "ae1ad05a2108484483d65a2d91426f33" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 33351 } } peers { permanent_uuid: "55f3851f9ad643c49db10785cd71553a" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 35085 } }
I20260811 18:55:07.063190 32562 tablet_replica.cc:439] T 3f50532b429d471291e1d41dcf368144 P ae1ad05a2108484483d65a2d91426f33: TxnStatusTablet state changed. Reason: New leader ae1ad05a2108484483d65a2d91426f33. Latest consensus state: current_term: 1 leader_uuid: "ae1ad05a2108484483d65a2d91426f33" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43665cdb5ae6441fbc7ffb7abed2cbaa" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 41133 } } peers { permanent_uuid: "ae1ad05a2108484483d65a2d91426f33" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 33351 } } peers { permanent_uuid: "55f3851f9ad643c49db10785cd71553a" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 35085 } } }
I20260811 18:55:07.063299 32562 tablet_replica.cc:442] T 3f50532b429d471291e1d41dcf368144 P ae1ad05a2108484483d65a2d91426f33: This TxnStatusTablet replica's current role is: LEADER
I20260811 18:55:07.063701 32587 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 18:55:07.063907 32587 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 18:55:07.064061 31592 catalog_manager.cc:5495] T 3f50532b429d471291e1d41dcf368144 P ae1ad05a2108484483d65a2d91426f33 reported cstate change: term changed from 0 to 1, leader changed from <none> to ae1ad05a2108484483d65a2d91426f33 (127.27.180.130). New cstate: current_term: 1 leader_uuid: "ae1ad05a2108484483d65a2d91426f33" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43665cdb5ae6441fbc7ffb7abed2cbaa" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 41133 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ae1ad05a2108484483d65a2d91426f33" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 33351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "55f3851f9ad643c49db10785cd71553a" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 35085 } health_report { overall_health: UNKNOWN } } }
I20260811 18:55:07.078639 32577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:07.074712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.130:33351 (local address 127.0.0.1:52824)
0811 18:55:07.074850 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:07.074865 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:07.074958 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:07.075275 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:07.075277 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:07.075284 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 18:55:07.075377 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:07.075381 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:07.076350 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:07.076352 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:07.076938 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:07.076946 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:07.077513 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:07.077516 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:07.077590 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:07.077636 (+ 46us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 18:55:07.078355 (+ 719us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 18:55:07.078373 (+ 18us) client_negotiation.cc:769] Sending connection context
0811 18:55:07.078449 (+ 76us) client_negotiation.cc:240] Negotiation successful
0811 18:55:07.078527 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260811 18:55:07.078662 32545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:07.074826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52824 (local address 127.27.180.130:33351)
0811 18:55:07.074956 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:07.074961 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:07.074977 (+ 16us) server_negotiation.cc:407] Connection header received
0811 18:55:07.075080 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:07.075083 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:07.075143 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:07.075250 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:55:07.075477 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:07.076240 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:07.077048 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:07.077410 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:07.077512 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:07.077771 (+ 259us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:55:07.078224 (+ 453us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:55:07.078347 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:07.078479 (+ 132us) server_negotiation.cc:299] Negotiation successful
0811 18:55:07.078536 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 18:55:07.083563 31908 raft_consensus.cc:1261] T 3f50532b429d471291e1d41dcf368144 P 43665cdb5ae6441fbc7ffb7abed2cbaa [term 1 FOLLOWER]: Refusing update from remote peer ae1ad05a2108484483d65a2d91426f33: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:55:07.083565 32421 raft_consensus.cc:1261] T 3f50532b429d471291e1d41dcf368144 P 55f3851f9ad643c49db10785cd71553a [term 1 FOLLOWER]: Refusing update from remote peer ae1ad05a2108484483d65a2d91426f33: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:55:07.084264 32560 consensus_queue.cc:1035] T 3f50532b429d471291e1d41dcf368144 P ae1ad05a2108484483d65a2d91426f33 [LEADER]: Connected to new peer: Peer: permanent_uuid: "55f3851f9ad643c49db10785cd71553a" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 35085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:55:07.084446 32562 consensus_queue.cc:1035] T 3f50532b429d471291e1d41dcf368144 P ae1ad05a2108484483d65a2d91426f33 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43665cdb5ae6441fbc7ffb7abed2cbaa" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 41133 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:55:07.087924 32559 tablet_replica.cc:439] T 3f50532b429d471291e1d41dcf368144 P 43665cdb5ae6441fbc7ffb7abed2cbaa: TxnStatusTablet state changed. Reason: New leader ae1ad05a2108484483d65a2d91426f33. Latest consensus state: current_term: 1 leader_uuid: "ae1ad05a2108484483d65a2d91426f33" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43665cdb5ae6441fbc7ffb7abed2cbaa" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 41133 } } peers { permanent_uuid: "ae1ad05a2108484483d65a2d91426f33" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 33351 } } peers { permanent_uuid: "55f3851f9ad643c49db10785cd71553a" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 35085 } } }
I20260811 18:55:07.088055 32559 tablet_replica.cc:442] T 3f50532b429d471291e1d41dcf368144 P 43665cdb5ae6441fbc7ffb7abed2cbaa: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 18:55:07.088467 32557 tablet_replica.cc:439] T 3f50532b429d471291e1d41dcf368144 P 55f3851f9ad643c49db10785cd71553a: TxnStatusTablet state changed. Reason: New leader ae1ad05a2108484483d65a2d91426f33. Latest consensus state: current_term: 1 leader_uuid: "ae1ad05a2108484483d65a2d91426f33" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43665cdb5ae6441fbc7ffb7abed2cbaa" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 41133 } } peers { permanent_uuid: "ae1ad05a2108484483d65a2d91426f33" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 33351 } } peers { permanent_uuid: "55f3851f9ad643c49db10785cd71553a" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 35085 } } }
I20260811 18:55:07.088600 32557 tablet_replica.cc:442] T 3f50532b429d471291e1d41dcf368144 P 55f3851f9ad643c49db10785cd71553a: This TxnStatusTablet replica's current role is: FOLLOWER
W20260811 18:55:07.089540 31988 tablet.cc:2369] T 3f50532b429d471291e1d41dcf368144 P 43665cdb5ae6441fbc7ffb7abed2cbaa: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:55:07.091375 32560 tablet_replica.cc:439] T 3f50532b429d471291e1d41dcf368144 P ae1ad05a2108484483d65a2d91426f33: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ae1ad05a2108484483d65a2d91426f33" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43665cdb5ae6441fbc7ffb7abed2cbaa" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 41133 } } peers { permanent_uuid: "ae1ad05a2108484483d65a2d91426f33" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 33351 } } peers { permanent_uuid: "55f3851f9ad643c49db10785cd71553a" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 35085 } } }
I20260811 18:55:07.091488 32560 tablet_replica.cc:442] T 3f50532b429d471291e1d41dcf368144 P ae1ad05a2108484483d65a2d91426f33: This TxnStatusTablet replica's current role is: LEADER
I20260811 18:55:07.093108 32559 tablet_replica.cc:439] T 3f50532b429d471291e1d41dcf368144 P 43665cdb5ae6441fbc7ffb7abed2cbaa: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ae1ad05a2108484483d65a2d91426f33" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43665cdb5ae6441fbc7ffb7abed2cbaa" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 41133 } } peers { permanent_uuid: "ae1ad05a2108484483d65a2d91426f33" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 33351 } } peers { permanent_uuid: "55f3851f9ad643c49db10785cd71553a" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 35085 } } }
I20260811 18:55:07.093226 32559 tablet_replica.cc:442] T 3f50532b429d471291e1d41dcf368144 P 43665cdb5ae6441fbc7ffb7abed2cbaa: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 18:55:07.097416 32562 tablet_replica.cc:439] T 3f50532b429d471291e1d41dcf368144 P ae1ad05a2108484483d65a2d91426f33: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ae1ad05a2108484483d65a2d91426f33" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43665cdb5ae6441fbc7ffb7abed2cbaa" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 41133 } } peers { permanent_uuid: "ae1ad05a2108484483d65a2d91426f33" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 33351 } } peers { permanent_uuid: "55f3851f9ad643c49db10785cd71553a" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 35085 } } }
I20260811 18:55:07.097522 32562 tablet_replica.cc:442] T 3f50532b429d471291e1d41dcf368144 P ae1ad05a2108484483d65a2d91426f33: This TxnStatusTablet replica's current role is: LEADER
I20260811 18:55:07.099154 32592 mvcc.cc:205] Tried to move back new op lower bound from 7317399581007409152 to 7317399580929490944. Current Snapshot: MvccSnapshot[applied={T|T < 7317399581007409152}]
I20260811 18:55:07.099344 32557 tablet_replica.cc:439] T 3f50532b429d471291e1d41dcf368144 P 55f3851f9ad643c49db10785cd71553a: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ae1ad05a2108484483d65a2d91426f33" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43665cdb5ae6441fbc7ffb7abed2cbaa" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 41133 } } peers { permanent_uuid: "ae1ad05a2108484483d65a2d91426f33" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 33351 } } peers { permanent_uuid: "55f3851f9ad643c49db10785cd71553a" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 35085 } } }
I20260811 18:55:07.099449 32557 tablet_replica.cc:442] T 3f50532b429d471291e1d41dcf368144 P 55f3851f9ad643c49db10785cd71553a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 18:55:07.101272 32545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:07.096963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52834 (local address 127.27.180.130:33351)
0811 18:55:07.097112 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:07.097117 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:07.097193 (+ 76us) server_negotiation.cc:407] Connection header received
0811 18:55:07.097381 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:07.097385 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:07.097437 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:07.097503 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:55:07.097843 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:07.098599 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:07.099817 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:07.100145 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:07.100229 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:07.100574 (+ 345us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:55:07.100651 (+ 77us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:55:07.100708 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:07.101099 (+ 391us) server_negotiation.cc:299] Negotiation successful
0811 18:55:07.101150 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 18:55:07.111486 32545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:07.108102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.130:35097 (local address 127.27.180.130:33351)
0811 18:55:07.108201 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:07.108205 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:07.108404 (+ 199us) server_negotiation.cc:407] Connection header received
0811 18:55:07.108485 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:07.108488 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:07.108532 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:07.108605 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:55:07.108863 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:07.109648 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:07.110468 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:07.110789 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:07.110882 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:07.111107 (+ 225us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:55:07.111175 (+ 68us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:55:07.111237 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:07.111314 (+ 77us) server_negotiation.cc:299] Negotiation successful
0811 18:55:07.111356 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":38}
I20260811 18:55:07.111488 32617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:07.108009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.130:33351 (local address 127.27.180.130:35097)
0811 18:55:07.108324 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:07.108338 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:07.108423 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:07.108613 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:07.108615 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:07.108620 (+ 5us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 18:55:07.108768 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:07.108774 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:07.109752 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:07.109755 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:07.110366 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:07.110375 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:07.110879 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:07.110881 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:07.110945 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:07.110985 (+ 40us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 18:55:07.111245 (+ 260us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 18:55:07.111250 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 18:55:07.111294 (+ 44us) client_negotiation.cc:240] Negotiation successful
0811 18:55:07.111352 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"mutex_wait_us":3,"thread_start_us":119,"threads_started":1}
I20260811 18:55:07.117586 32421 raft_consensus.cc:1261] T d7ff45b6bf794be2b5f0beb8e619f54c P 55f3851f9ad643c49db10785cd71553a [term 1 FOLLOWER]: Refusing update from remote peer ae1ad05a2108484483d65a2d91426f33: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:55:07.117919 31908 raft_consensus.cc:1261] T d7ff45b6bf794be2b5f0beb8e619f54c P 43665cdb5ae6441fbc7ffb7abed2cbaa [term 1 FOLLOWER]: Refusing update from remote peer ae1ad05a2108484483d65a2d91426f33: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:55:07.117998 32562 consensus_queue.cc:1035] T d7ff45b6bf794be2b5f0beb8e619f54c P ae1ad05a2108484483d65a2d91426f33 [LEADER]: Connected to new peer: Peer: permanent_uuid: "55f3851f9ad643c49db10785cd71553a" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 35085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:55:07.118306 32562 consensus_queue.cc:1035] T d7ff45b6bf794be2b5f0beb8e619f54c P ae1ad05a2108484483d65a2d91426f33 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43665cdb5ae6441fbc7ffb7abed2cbaa" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 41133 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:55:07.121513 32599 mvcc.cc:205] Tried to move back new op lower bound from 7317399581149798400 to 7317399580553887744. Current Snapshot: MvccSnapshot[applied={T|T < 7317399581149798400}]
I20260811 18:55:07.125140 32591 mvcc.cc:205] Tried to move back new op lower bound from 7317399581149798400 to 7317399580553887744. Current Snapshot: MvccSnapshot[applied={T|T < 7317399581149798400}]
I20260811 18:55:07.156230 31592 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42148:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:55:07.156386 31592 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42148:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:55:07.158619 31592 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3b87a006e181434094f7d5896eb20643: Sending DeleteTablet for 3 replicas of tablet d7ff45b6bf794be2b5f0beb8e619f54c
I20260811 18:55:07.159325 31862 tablet_service.cc:1514] Processing DeleteTablet for tablet d7ff45b6bf794be2b5f0beb8e619f54c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:55:07 UTC) from {username='kudu', principal='kudu/127.27.180.190@KRBTEST.COM'} at 127.0.0.1:51424
I20260811 18:55:07.160013 32623 tablet_replica.cc:330] T d7ff45b6bf794be2b5f0beb8e619f54c P 43665cdb5ae6441fbc7ffb7abed2cbaa: stopping tablet replica
I20260811 18:55:07.160171 32400 tablet_service.cc:1514] Processing DeleteTablet for tablet d7ff45b6bf794be2b5f0beb8e619f54c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:55:07 UTC) from {username='kudu', principal='kudu/127.27.180.190@KRBTEST.COM'} at 127.0.0.1:48190
I20260811 18:55:07.160216 31592 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42148:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:55:07.160435 32623 raft_consensus.cc:2229] T d7ff45b6bf794be2b5f0beb8e619f54c P 43665cdb5ae6441fbc7ffb7abed2cbaa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:55:07.160641 32623 raft_consensus.cc:2258] T d7ff45b6bf794be2b5f0beb8e619f54c P 43665cdb5ae6441fbc7ffb7abed2cbaa [term 1 FOLLOWER]: Raft consensus is shut down!
W20260811 18:55:07.160637 31592 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:55:07.160902 32136 tablet_service.cc:1514] Processing DeleteTablet for tablet d7ff45b6bf794be2b5f0beb8e619f54c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:55:07 UTC) from {username='kudu', principal='kudu/127.27.180.190@KRBTEST.COM'} at 127.0.0.1:52818
I20260811 18:55:07.161827 32626 tablet_replica.cc:330] T d7ff45b6bf794be2b5f0beb8e619f54c P 55f3851f9ad643c49db10785cd71553a: stopping tablet replica
I20260811 18:55:07.162163 32626 raft_consensus.cc:2229] T d7ff45b6bf794be2b5f0beb8e619f54c P 55f3851f9ad643c49db10785cd71553a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:55:07.162277 32623 ts_tablet_manager.cc:1905] T d7ff45b6bf794be2b5f0beb8e619f54c P 43665cdb5ae6441fbc7ffb7abed2cbaa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:55:07.162416 32626 raft_consensus.cc:2258] T d7ff45b6bf794be2b5f0beb8e619f54c P 55f3851f9ad643c49db10785cd71553a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:55:07.162798 32627 tablet_replica.cc:330] T d7ff45b6bf794be2b5f0beb8e619f54c P ae1ad05a2108484483d65a2d91426f33: stopping tablet replica
I20260811 18:55:07.163086 32627 raft_consensus.cc:2229] T d7ff45b6bf794be2b5f0beb8e619f54c P ae1ad05a2108484483d65a2d91426f33 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:55:07.163440 32627 raft_consensus.cc:2258] T d7ff45b6bf794be2b5f0beb8e619f54c P ae1ad05a2108484483d65a2d91426f33 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:55:07.165066 32626 ts_tablet_manager.cc:1905] T d7ff45b6bf794be2b5f0beb8e619f54c P 55f3851f9ad643c49db10785cd71553a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:55:07.166105 32136 tablet_service.cc:1467] Processing CreateTablet for tablet aab8f5edf3ae4ba8b7066b2bbac9c56e (DEFAULT_TABLE table=test-table [id=39a47c9482fc45f1af41bacd3582d8a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:55:07.166390 32136 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aab8f5edf3ae4ba8b7066b2bbac9c56e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:55:07.166739 32623 ts_tablet_manager.cc:1918] T d7ff45b6bf794be2b5f0beb8e619f54c P 43665cdb5ae6441fbc7ffb7abed2cbaa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 18:55:07.166870 32623 log.cc:1196] T d7ff45b6bf794be2b5f0beb8e619f54c P 43665cdb5ae6441fbc7ffb7abed2cbaa: Deleting WAL directory at /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/minicluster-data/ts-0/wal/wals/d7ff45b6bf794be2b5f0beb8e619f54c
I20260811 18:55:07.167955 32623 ts_tablet_manager.cc:1939] T d7ff45b6bf794be2b5f0beb8e619f54c P 43665cdb5ae6441fbc7ffb7abed2cbaa: Deleting consensus metadata
I20260811 18:55:07.168111 32626 ts_tablet_manager.cc:1918] T d7ff45b6bf794be2b5f0beb8e619f54c P 55f3851f9ad643c49db10785cd71553a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 18:55:07.168191 32626 log.cc:1196] T d7ff45b6bf794be2b5f0beb8e619f54c P 55f3851f9ad643c49db10785cd71553a: Deleting WAL directory at /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/minicluster-data/ts-2/wal/wals/d7ff45b6bf794be2b5f0beb8e619f54c
I20260811 18:55:07.168484 32626 ts_tablet_manager.cc:1939] T d7ff45b6bf794be2b5f0beb8e619f54c P 55f3851f9ad643c49db10785cd71553a: Deleting consensus metadata
I20260811 18:55:07.169013 32400 tablet_service.cc:1467] Processing CreateTablet for tablet aab8f5edf3ae4ba8b7066b2bbac9c56e (DEFAULT_TABLE table=test-table [id=39a47c9482fc45f1af41bacd3582d8a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:55:07.169167 31568 catalog_manager.cc:4841] TS 43665cdb5ae6441fbc7ffb7abed2cbaa (127.27.180.129:41133): tablet d7ff45b6bf794be2b5f0beb8e619f54c (table test-table [id=7adb8e566f3546c8b29f671e7d142952]) successfully deleted
I20260811 18:55:07.169276 32400 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aab8f5edf3ae4ba8b7066b2bbac9c56e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:55:07.169689 32548 tablet_bootstrap.cc:492] T aab8f5edf3ae4ba8b7066b2bbac9c56e P ae1ad05a2108484483d65a2d91426f33: Bootstrap starting.
I20260811 18:55:07.170483 32548 tablet_bootstrap.cc:654] T aab8f5edf3ae4ba8b7066b2bbac9c56e P ae1ad05a2108484483d65a2d91426f33: Neither blocks nor log segments found. Creating new log.
I20260811 18:55:07.171188 32551 tablet_bootstrap.cc:492] T aab8f5edf3ae4ba8b7066b2bbac9c56e P 55f3851f9ad643c49db10785cd71553a: Bootstrap starting.
I20260811 18:55:07.171480 31862 tablet_service.cc:1467] Processing CreateTablet for tablet aab8f5edf3ae4ba8b7066b2bbac9c56e (DEFAULT_TABLE table=test-table [id=39a47c9482fc45f1af41bacd3582d8a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:55:07.171720 31862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aab8f5edf3ae4ba8b7066b2bbac9c56e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:55:07.171924 32551 tablet_bootstrap.cc:654] T aab8f5edf3ae4ba8b7066b2bbac9c56e P 55f3851f9ad643c49db10785cd71553a: Neither blocks nor log segments found. Creating new log.
I20260811 18:55:07.172582 31574 catalog_manager.cc:4841] TS 55f3851f9ad643c49db10785cd71553a (127.27.180.131:35085): tablet d7ff45b6bf794be2b5f0beb8e619f54c (table test-table [id=7adb8e566f3546c8b29f671e7d142952]) successfully deleted
I20260811 18:55:07.173480 32551 tablet_bootstrap.cc:492] T aab8f5edf3ae4ba8b7066b2bbac9c56e P 55f3851f9ad643c49db10785cd71553a: No bootstrap required, opened a new log
I20260811 18:55:07.173560 32551 ts_tablet_manager.cc:1397] T aab8f5edf3ae4ba8b7066b2bbac9c56e P 55f3851f9ad643c49db10785cd71553a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:55:07.173898 32551 raft_consensus.cc:348] T aab8f5edf3ae4ba8b7066b2bbac9c56e P 55f3851f9ad643c49db10785cd71553a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43665cdb5ae6441fbc7ffb7abed2cbaa" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 41133 } } peers { permanent_uuid: "ae1ad05a2108484483d65a2d91426f33" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 33351 } } peers { permanent_uuid: "55f3851f9ad643c49db10785cd71553a" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 35085 } }
I20260811 18:55:07.174018 32546 tablet_bootstrap.cc:492] T aab8f5edf3ae4ba8b7066b2bbac9c56e P 43665cdb5ae6441fbc7ffb7abed2cbaa: Bootstrap starting.
I20260811 18:55:07.174018 32551 raft_consensus.cc:374] T aab8f5edf3ae4ba8b7066b2bbac9c56e P 55f3851f9ad643c49db10785cd71553a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:55:07.174052 32551 raft_consensus.cc:729] T aab8f5edf3ae4ba8b7066b2bbac9c56e P 55f3851f9ad643c49db10785cd71553a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55f3851f9ad643c49db10785cd71553a, State: Initialized, Role: FOLLOWER
I20260811 18:55:07.174167 32551 consensus_queue.cc:260] T aab8f5edf3ae4ba8b7066b2bbac9c56e P 55f3851f9ad643c49db10785cd71553a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43665cdb5ae6441fbc7ffb7abed2cbaa" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 41133 } } peers { permanent_uuid: "ae1ad05a2108484483d65a2d91426f33" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 33351 } } peers { permanent_uuid: "55f3851f9ad643c49db10785cd71553a" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 35085 } }
I20260811 18:55:07.174418 32548 tablet_bootstrap.cc:492] T aab8f5edf3ae4ba8b7066b2bbac9c56e P ae1ad05a2108484483d65a2d91426f33: No bootstrap required, opened a new log
I20260811 18:55:07.174428 32551 ts_tablet_manager.cc:1428] T aab8f5edf3ae4ba8b7066b2bbac9c56e P 55f3851f9ad643c49db10785cd71553a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 18:55:07.174487 32548 ts_tablet_manager.cc:1397] T aab8f5edf3ae4ba8b7066b2bbac9c56e P ae1ad05a2108484483d65a2d91426f33: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260811 18:55:07.174827 32548 raft_consensus.cc:348] T aab8f5edf3ae4ba8b7066b2bbac9c56e P ae1ad05a2108484483d65a2d91426f33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43665cdb5ae6441fbc7ffb7abed2cbaa" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 41133 } } peers { permanent_uuid: "ae1ad05a2108484483d65a2d91426f33" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 33351 } } peers { permanent_uuid: "55f3851f9ad643c49db10785cd71553a" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 35085 } }
I20260811 18:55:07.174938 32548 raft_consensus.cc:374] T aab8f5edf3ae4ba8b7066b2bbac9c56e P ae1ad05a2108484483d65a2d91426f33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:55:07.174968 32548 raft_consensus.cc:729] T aab8f5edf3ae4ba8b7066b2bbac9c56e P ae1ad05a2108484483d65a2d91426f33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae1ad05a2108484483d65a2d91426f33, State: Initialized, Role: FOLLOWER
I20260811 18:55:07.175048 32546 tablet_bootstrap.cc:654] T aab8f5edf3ae4ba8b7066b2bbac9c56e P 43665cdb5ae6441fbc7ffb7abed2cbaa: Neither blocks nor log segments found. Creating new log.
I20260811 18:55:07.175072 32548 consensus_queue.cc:260] T aab8f5edf3ae4ba8b7066b2bbac9c56e P ae1ad05a2108484483d65a2d91426f33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43665cdb5ae6441fbc7ffb7abed2cbaa" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 41133 } } peers { permanent_uuid: "ae1ad05a2108484483d65a2d91426f33" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 33351 } } peers { permanent_uuid: "55f3851f9ad643c49db10785cd71553a" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 35085 } }
I20260811 18:55:07.175319 32548 ts_tablet_manager.cc:1428] T aab8f5edf3ae4ba8b7066b2bbac9c56e P ae1ad05a2108484483d65a2d91426f33: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 18:55:07.176959 32627 ts_tablet_manager.cc:1905] T d7ff45b6bf794be2b5f0beb8e619f54c P ae1ad05a2108484483d65a2d91426f33: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:55:07.177457 32546 tablet_bootstrap.cc:492] T aab8f5edf3ae4ba8b7066b2bbac9c56e P 43665cdb5ae6441fbc7ffb7abed2cbaa: No bootstrap required, opened a new log
I20260811 18:55:07.177551 32546 ts_tablet_manager.cc:1397] T aab8f5edf3ae4ba8b7066b2bbac9c56e P 43665cdb5ae6441fbc7ffb7abed2cbaa: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260811 18:55:07.178004 32546 raft_consensus.cc:348] T aab8f5edf3ae4ba8b7066b2bbac9c56e P 43665cdb5ae6441fbc7ffb7abed2cbaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43665cdb5ae6441fbc7ffb7abed2cbaa" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 41133 } } peers { permanent_uuid: "ae1ad05a2108484483d65a2d91426f33" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 33351 } } peers { permanent_uuid: "55f3851f9ad643c49db10785cd71553a" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 35085 } }
I20260811 18:55:07.178125 32546 raft_consensus.cc:374] T aab8f5edf3ae4ba8b7066b2bbac9c56e P 43665cdb5ae6441fbc7ffb7abed2cbaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:55:07.178166 32546 raft_consensus.cc:729] T aab8f5edf3ae4ba8b7066b2bbac9c56e P 43665cdb5ae6441fbc7ffb7abed2cbaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43665cdb5ae6441fbc7ffb7abed2cbaa, State: Initialized, Role: FOLLOWER
I20260811 18:55:07.178294 32546 consensus_queue.cc:260] T aab8f5edf3ae4ba8b7066b2bbac9c56e P 43665cdb5ae6441fbc7ffb7abed2cbaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43665cdb5ae6441fbc7ffb7abed2cbaa" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 41133 } } peers { permanent_uuid: "ae1ad05a2108484483d65a2d91426f33" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 33351 } } peers { permanent_uuid: "55f3851f9ad643c49db10785cd71553a" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 35085 } }
I20260811 18:55:07.178568 32546 ts_tablet_manager.cc:1428] T aab8f5edf3ae4ba8b7066b2bbac9c56e P 43665cdb5ae6441fbc7ffb7abed2cbaa: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 18:55:07.180044 32627 ts_tablet_manager.cc:1918] T d7ff45b6bf794be2b5f0beb8e619f54c P ae1ad05a2108484483d65a2d91426f33: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 18:55:07.180142 32627 log.cc:1196] T d7ff45b6bf794be2b5f0beb8e619f54c P ae1ad05a2108484483d65a2d91426f33: Deleting WAL directory at /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/minicluster-data/ts-1/wal/wals/d7ff45b6bf794be2b5f0beb8e619f54c
I20260811 18:55:07.180457 32627 ts_tablet_manager.cc:1939] T d7ff45b6bf794be2b5f0beb8e619f54c P ae1ad05a2108484483d65a2d91426f33: Deleting consensus metadata
I20260811 18:55:07.181495 31574 catalog_manager.cc:4841] TS ae1ad05a2108484483d65a2d91426f33 (127.27.180.130:33351): tablet d7ff45b6bf794be2b5f0beb8e619f54c (table test-table [id=7adb8e566f3546c8b29f671e7d142952]) successfully deleted
I20260811 18:55:07.204217 32557 raft_consensus.cc:482] T aab8f5edf3ae4ba8b7066b2bbac9c56e P 55f3851f9ad643c49db10785cd71553a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:55:07.204367 32557 raft_consensus.cc:504] T aab8f5edf3ae4ba8b7066b2bbac9c56e P 55f3851f9ad643c49db10785cd71553a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43665cdb5ae6441fbc7ffb7abed2cbaa" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 41133 } } peers { permanent_uuid: "ae1ad05a2108484483d65a2d91426f33" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 33351 } } peers { permanent_uuid: "55f3851f9ad643c49db10785cd71553a" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 35085 } }
I20260811 18:55:07.206041 32557 leader_election.cc:288] T aab8f5edf3ae4ba8b7066b2bbac9c56e P 55f3851f9ad643c49db10785cd71553a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43665cdb5ae6441fbc7ffb7abed2cbaa (127.27.180.129:41133), ae1ad05a2108484483d65a2d91426f33 (127.27.180.130:33351)
I20260811 18:55:07.210700 32315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:07.206267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.129:41133 (local address 127.27.180.131:56619)
0811 18:55:07.206621 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:07.206637 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:07.206736 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:07.207138 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:07.207140 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:07.207159 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:55:07.207277 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:07.207284 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:07.208166 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:07.208169 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:07.209444 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:07.209455 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:07.210286 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:07.210290 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:07.210396 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:07.210415 (+ 19us) client_negotiation.cc:769] Sending connection context
0811 18:55:07.210478 (+ 63us) client_negotiation.cc:240] Negotiation successful
0811 18:55:07.210548 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283}
I20260811 18:55:07.210691 32543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:07.206439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.131:56619 (local address 127.27.180.129:41133)
0811 18:55:07.206592 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:07.206596 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:07.206724 (+ 128us) server_negotiation.cc:407] Connection header received
0811 18:55:07.206889 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:07.206893 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:07.206949 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:07.207012 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:55:07.207418 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:07.208070 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:07.209581 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:07.210166 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:07.210275 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:07.210331 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:07.210514 (+ 183us) server_negotiation.cc:299] Negotiation successful
0811 18:55:07.210588 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 18:55:07.211158 31908 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aab8f5edf3ae4ba8b7066b2bbac9c56e" candidate_uuid: "55f3851f9ad643c49db10785cd71553a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43665cdb5ae6441fbc7ffb7abed2cbaa" is_pre_election: true
I20260811 18:55:07.211292 32545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:07.206776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.131:57555 (local address 127.27.180.130:33351)
0811 18:55:07.206897 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:07.206901 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:07.207462 (+ 561us) server_negotiation.cc:407] Connection header received
0811 18:55:07.207675 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:07.207678 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:07.207731 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:07.207817 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:55:07.208089 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:07.208701 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:07.210049 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:07.210645 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:07.210754 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:07.210801 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:07.211098 (+ 297us) server_negotiation.cc:299] Negotiation successful
0811 18:55:07.211168 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 18:55:07.211318 31908 raft_consensus.cc:2454] T aab8f5edf3ae4ba8b7066b2bbac9c56e P 43665cdb5ae6441fbc7ffb7abed2cbaa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 55f3851f9ad643c49db10785cd71553a in term 0.
I20260811 18:55:07.211704 32635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:07.206366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.130:33351 (local address 127.27.180.131:57555)
0811 18:55:07.207369 (+ 1003us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:07.207384 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:07.207527 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:07.207841 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:07.207844 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:07.207860 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:55:07.207988 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:07.207994 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:07.208822 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:07.208826 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:07.209923 (+ 1097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:07.209934 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:07.210863 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:07.210867 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:07.210974 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:07.210997 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 18:55:07.211493 (+ 496us) client_negotiation.cc:240] Negotiation successful
0811 18:55:07.211566 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":653,"thread_start_us":95,"threads_started":1}
I20260811 18:55:07.211673 32306 leader_election.cc:302] T aab8f5edf3ae4ba8b7066b2bbac9c56e P 55f3851f9ad643c49db10785cd71553a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43665cdb5ae6441fbc7ffb7abed2cbaa, 55f3851f9ad643c49db10785cd71553a; no voters:
I20260811 18:55:07.211966 32557 raft_consensus.cc:2781] T aab8f5edf3ae4ba8b7066b2bbac9c56e P 55f3851f9ad643c49db10785cd71553a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:55:07.212020 32557 raft_consensus.cc:482] T aab8f5edf3ae4ba8b7066b2bbac9c56e P 55f3851f9ad643c49db10785cd71553a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:55:07.212060 32557 raft_consensus.cc:3037] T aab8f5edf3ae4ba8b7066b2bbac9c56e P 55f3851f9ad643c49db10785cd71553a [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:55:07.212352 32179 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aab8f5edf3ae4ba8b7066b2bbac9c56e" candidate_uuid: "55f3851f9ad643c49db10785cd71553a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae1ad05a2108484483d65a2d91426f33" is_pre_election: true
I20260811 18:55:07.212572 32179 raft_consensus.cc:2454] T aab8f5edf3ae4ba8b7066b2bbac9c56e P ae1ad05a2108484483d65a2d91426f33 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 55f3851f9ad643c49db10785cd71553a in term 0.
I20260811 18:55:07.213272 32557 raft_consensus.cc:504] T aab8f5edf3ae4ba8b7066b2bbac9c56e P 55f3851f9ad643c49db10785cd71553a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43665cdb5ae6441fbc7ffb7abed2cbaa" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 41133 } } peers { permanent_uuid: "ae1ad05a2108484483d65a2d91426f33" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 33351 } } peers { permanent_uuid: "55f3851f9ad643c49db10785cd71553a" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 35085 } }
I20260811 18:55:07.213608 32557 leader_election.cc:288] T aab8f5edf3ae4ba8b7066b2bbac9c56e P 55f3851f9ad643c49db10785cd71553a [CANDIDATE]: Term 1 election: Requested vote from peers 43665cdb5ae6441fbc7ffb7abed2cbaa (127.27.180.129:41133), ae1ad05a2108484483d65a2d91426f33 (127.27.180.130:33351)
I20260811 18:55:07.213949 31908 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aab8f5edf3ae4ba8b7066b2bbac9c56e" candidate_uuid: "55f3851f9ad643c49db10785cd71553a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43665cdb5ae6441fbc7ffb7abed2cbaa"
I20260811 18:55:07.213972 32179 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aab8f5edf3ae4ba8b7066b2bbac9c56e" candidate_uuid: "55f3851f9ad643c49db10785cd71553a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae1ad05a2108484483d65a2d91426f33"
I20260811 18:55:07.214063 31908 raft_consensus.cc:3037] T aab8f5edf3ae4ba8b7066b2bbac9c56e P 43665cdb5ae6441fbc7ffb7abed2cbaa [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:55:07.214063 32179 raft_consensus.cc:3037] T aab8f5edf3ae4ba8b7066b2bbac9c56e P ae1ad05a2108484483d65a2d91426f33 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:55:07.215119 31908 raft_consensus.cc:2454] T aab8f5edf3ae4ba8b7066b2bbac9c56e P 43665cdb5ae6441fbc7ffb7abed2cbaa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55f3851f9ad643c49db10785cd71553a in term 1.
I20260811 18:55:07.215124 32179 raft_consensus.cc:2454] T aab8f5edf3ae4ba8b7066b2bbac9c56e P ae1ad05a2108484483d65a2d91426f33 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55f3851f9ad643c49db10785cd71553a in term 1.
I20260811 18:55:07.215423 32308 leader_election.cc:302] T aab8f5edf3ae4ba8b7066b2bbac9c56e P 55f3851f9ad643c49db10785cd71553a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55f3851f9ad643c49db10785cd71553a, ae1ad05a2108484483d65a2d91426f33; no voters:
I20260811 18:55:07.215677 32557 raft_consensus.cc:2781] T aab8f5edf3ae4ba8b7066b2bbac9c56e P 55f3851f9ad643c49db10785cd71553a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:55:07.215864 32557 raft_consensus.cc:686] T aab8f5edf3ae4ba8b7066b2bbac9c56e P 55f3851f9ad643c49db10785cd71553a [term 1 LEADER]: Becoming Leader. State: Replica: 55f3851f9ad643c49db10785cd71553a, State: Running, Role: LEADER
I20260811 18:55:07.216107 32557 consensus_queue.cc:237] T aab8f5edf3ae4ba8b7066b2bbac9c56e P 55f3851f9ad643c49db10785cd71553a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43665cdb5ae6441fbc7ffb7abed2cbaa" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 41133 } } peers { permanent_uuid: "ae1ad05a2108484483d65a2d91426f33" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 33351 } } peers { permanent_uuid: "55f3851f9ad643c49db10785cd71553a" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 35085 } }
I20260811 18:55:07.218324 31592 catalog_manager.cc:5495] T aab8f5edf3ae4ba8b7066b2bbac9c56e P 55f3851f9ad643c49db10785cd71553a reported cstate change: term changed from 0 to 1, leader changed from <none> to 55f3851f9ad643c49db10785cd71553a (127.27.180.131). New cstate: current_term: 1 leader_uuid: "55f3851f9ad643c49db10785cd71553a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43665cdb5ae6441fbc7ffb7abed2cbaa" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 41133 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ae1ad05a2108484483d65a2d91426f33" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 33351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "55f3851f9ad643c49db10785cd71553a" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 35085 } health_report { overall_health: HEALTHY } } }
I20260811 18:55:07.243770 32534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:07.240021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48196 (local address 127.27.180.131:35085)
0811 18:55:07.240130 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:07.240134 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:07.240146 (+ 12us) server_negotiation.cc:407] Connection header received
0811 18:55:07.240299 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:07.240301 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:07.240350 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:07.240435 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:55:07.240646 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:07.241239 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:07.242399 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:07.242643 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:07.242722 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:07.242964 (+ 242us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:55:07.243429 (+ 465us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:55:07.243514 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:07.243590 (+ 76us) server_negotiation.cc:299] Negotiation successful
0811 18:55:07.243637 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 18:55:07.253720 32514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:07.250620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.130:33351 (local address 127.27.180.131:60299)
0811 18:55:07.250771 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:07.250787 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:07.250873 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:07.251092 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:07.251094 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:07.251101 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 18:55:07.251206 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:07.251211 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:07.251924 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:07.251927 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:07.252530 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:07.252538 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:07.252980 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:07.252983 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:07.253067 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:07.253142 (+ 75us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 18:55:07.253430 (+ 288us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 18:55:07.253437 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 18:55:07.253498 (+ 61us) client_negotiation.cc:240] Negotiation successful
0811 18:55:07.253579 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260811 18:55:07.253720 32545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:07.250752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.131:60299 (local address 127.27.180.130:33351)
0811 18:55:07.250895 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:07.250900 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:07.250914 (+ 14us) server_negotiation.cc:407] Connection header received
0811 18:55:07.250963 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:07.250966 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:07.251007 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:07.251066 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:55:07.251322 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:07.251812 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:07.252652 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:07.252870 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:07.252958 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:07.253275 (+ 317us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:55:07.253345 (+ 70us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:55:07.253406 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:07.253525 (+ 119us) server_negotiation.cc:299] Negotiation successful
0811 18:55:07.253575 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 18:55:07.260593 31908 raft_consensus.cc:1261] T aab8f5edf3ae4ba8b7066b2bbac9c56e P 43665cdb5ae6441fbc7ffb7abed2cbaa [term 1 FOLLOWER]: Refusing update from remote peer 55f3851f9ad643c49db10785cd71553a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:55:07.260903 32179 raft_consensus.cc:1261] T aab8f5edf3ae4ba8b7066b2bbac9c56e P ae1ad05a2108484483d65a2d91426f33 [term 1 FOLLOWER]: Refusing update from remote peer 55f3851f9ad643c49db10785cd71553a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:55:07.261061 32557 consensus_queue.cc:1035] T aab8f5edf3ae4ba8b7066b2bbac9c56e P 55f3851f9ad643c49db10785cd71553a [LEADER]: Connected to new peer: Peer: permanent_uuid: "43665cdb5ae6441fbc7ffb7abed2cbaa" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 41133 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:55:07.261435 32637 consensus_queue.cc:1035] T aab8f5edf3ae4ba8b7066b2bbac9c56e P 55f3851f9ad643c49db10785cd71553a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ae1ad05a2108484483d65a2d91426f33" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 33351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:55:07.280905 32534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:07.277313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.130:37923 (local address 127.27.180.131:35085)
0811 18:55:07.277713 (+ 400us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:07.277717 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:07.277736 (+ 19us) server_negotiation.cc:407] Connection header received
0811 18:55:07.277769 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:07.277771 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:07.277809 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:07.277864 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:55:07.278151 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:07.278643 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:07.279780 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:07.280114 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:07.280209 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:07.280469 (+ 260us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:55:07.280547 (+ 78us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:55:07.280609 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:07.280745 (+ 136us) server_negotiation.cc:299] Negotiation successful
0811 18:55:07.280801 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343}
I20260811 18:55:07.280934 32617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:07.276792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.131:35085 (local address 127.27.180.130:37923)
0811 18:55:07.277026 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:07.277042 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:07.277173 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:07.277907 (+ 734us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:07.277910 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:07.277919 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 18:55:07.278021 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:07.278028 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:07.278737 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:07.278741 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:07.279647 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:07.279656 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:07.280232 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:07.280236 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:07.280325 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:07.280360 (+ 35us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 18:55:07.280650 (+ 290us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 18:55:07.280657 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 18:55:07.280719 (+ 62us) client_negotiation.cc:240] Negotiation successful
0811 18:55:07.280799 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144}
W20260811 18:55:07.281646 32592 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260811 18:55:08.842458 31600 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42148:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:55:08.842620 31600 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42148:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:55:08.844990 31600 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3b87a006e181434094f7d5896eb20643: Sending DeleteTablet for 3 replicas of tablet aab8f5edf3ae4ba8b7066b2bbac9c56e
I20260811 18:55:08.845508 32136 tablet_service.cc:1514] Processing DeleteTablet for tablet aab8f5edf3ae4ba8b7066b2bbac9c56e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:55:08 UTC) from {username='kudu', principal='kudu/127.27.180.190@KRBTEST.COM'} at 127.0.0.1:52818
I20260811 18:55:08.845521 31862 tablet_service.cc:1514] Processing DeleteTablet for tablet aab8f5edf3ae4ba8b7066b2bbac9c56e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:55:08 UTC) from {username='kudu', principal='kudu/127.27.180.190@KRBTEST.COM'} at 127.0.0.1:51424
I20260811 18:55:08.845644 32400 tablet_service.cc:1514] Processing DeleteTablet for tablet aab8f5edf3ae4ba8b7066b2bbac9c56e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:55:08 UTC) from {username='kudu', principal='kudu/127.27.180.190@KRBTEST.COM'} at 127.0.0.1:48190
I20260811 18:55:08.845846 32691 tablet_replica.cc:330] T aab8f5edf3ae4ba8b7066b2bbac9c56e P 43665cdb5ae6441fbc7ffb7abed2cbaa: stopping tablet replica
I20260811 18:55:08.845966 32693 tablet_replica.cc:330] T aab8f5edf3ae4ba8b7066b2bbac9c56e P 55f3851f9ad643c49db10785cd71553a: stopping tablet replica
I20260811 18:55:08.845986 32691 raft_consensus.cc:2229] T aab8f5edf3ae4ba8b7066b2bbac9c56e P 43665cdb5ae6441fbc7ffb7abed2cbaa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:55:08.846094 32691 raft_consensus.cc:2258] T aab8f5edf3ae4ba8b7066b2bbac9c56e P 43665cdb5ae6441fbc7ffb7abed2cbaa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:55:08.846098 32692 tablet_replica.cc:330] T aab8f5edf3ae4ba8b7066b2bbac9c56e P ae1ad05a2108484483d65a2d91426f33: stopping tablet replica
I20260811 18:55:08.846132 32693 raft_consensus.cc:2229] T aab8f5edf3ae4ba8b7066b2bbac9c56e P 55f3851f9ad643c49db10785cd71553a [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:55:08.846201 32692 raft_consensus.cc:2229] T aab8f5edf3ae4ba8b7066b2bbac9c56e P ae1ad05a2108484483d65a2d91426f33 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:55:08.846318 32692 raft_consensus.cc:2258] T aab8f5edf3ae4ba8b7066b2bbac9c56e P ae1ad05a2108484483d65a2d91426f33 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:55:08.846467 32693 raft_consensus.cc:2258] T aab8f5edf3ae4ba8b7066b2bbac9c56e P 55f3851f9ad643c49db10785cd71553a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:55:08.847046 32693 ts_tablet_manager.cc:1905] T aab8f5edf3ae4ba8b7066b2bbac9c56e P 55f3851f9ad643c49db10785cd71553a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:55:08.847290 32692 ts_tablet_manager.cc:1905] T aab8f5edf3ae4ba8b7066b2bbac9c56e P ae1ad05a2108484483d65a2d91426f33: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:55:08.849772 32692 ts_tablet_manager.cc:1918] T aab8f5edf3ae4ba8b7066b2bbac9c56e P ae1ad05a2108484483d65a2d91426f33: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 18:55:08.849772 32693 ts_tablet_manager.cc:1918] T aab8f5edf3ae4ba8b7066b2bbac9c56e P 55f3851f9ad643c49db10785cd71553a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 18:55:08.849852 32693 log.cc:1196] T aab8f5edf3ae4ba8b7066b2bbac9c56e P 55f3851f9ad643c49db10785cd71553a: Deleting WAL directory at /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/minicluster-data/ts-2/wal/wals/aab8f5edf3ae4ba8b7066b2bbac9c56e
I20260811 18:55:08.849876 32692 log.cc:1196] T aab8f5edf3ae4ba8b7066b2bbac9c56e P ae1ad05a2108484483d65a2d91426f33: Deleting WAL directory at /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/minicluster-data/ts-1/wal/wals/aab8f5edf3ae4ba8b7066b2bbac9c56e
I20260811 18:55:08.850140 32692 ts_tablet_manager.cc:1939] T aab8f5edf3ae4ba8b7066b2bbac9c56e P ae1ad05a2108484483d65a2d91426f33: Deleting consensus metadata
I20260811 18:55:08.850140 32693 ts_tablet_manager.cc:1939] T aab8f5edf3ae4ba8b7066b2bbac9c56e P 55f3851f9ad643c49db10785cd71553a: Deleting consensus metadata
I20260811 18:55:08.850368 32691 ts_tablet_manager.cc:1905] T aab8f5edf3ae4ba8b7066b2bbac9c56e P 43665cdb5ae6441fbc7ffb7abed2cbaa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:55:08.850780 31574 catalog_manager.cc:4841] TS ae1ad05a2108484483d65a2d91426f33 (127.27.180.130:33351): tablet aab8f5edf3ae4ba8b7066b2bbac9c56e (table test-table [id=39a47c9482fc45f1af41bacd3582d8a1]) successfully deleted
I20260811 18:55:08.850944 31574 catalog_manager.cc:4841] TS 55f3851f9ad643c49db10785cd71553a (127.27.180.131:35085): tablet aab8f5edf3ae4ba8b7066b2bbac9c56e (table test-table [id=39a47c9482fc45f1af41bacd3582d8a1]) successfully deleted
I20260811 18:55:08.852881 32691 ts_tablet_manager.cc:1918] T aab8f5edf3ae4ba8b7066b2bbac9c56e P 43665cdb5ae6441fbc7ffb7abed2cbaa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 18:55:08.852947 32691 log.cc:1196] T aab8f5edf3ae4ba8b7066b2bbac9c56e P 43665cdb5ae6441fbc7ffb7abed2cbaa: Deleting WAL directory at /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/minicluster-data/ts-0/wal/wals/aab8f5edf3ae4ba8b7066b2bbac9c56e
I20260811 18:55:08.853180 32691 ts_tablet_manager.cc:1939] T aab8f5edf3ae4ba8b7066b2bbac9c56e P 43665cdb5ae6441fbc7ffb7abed2cbaa: Deleting consensus metadata
I20260811 18:55:08.853783 31568 catalog_manager.cc:4841] TS 43665cdb5ae6441fbc7ffb7abed2cbaa (127.27.180.129:41133): tablet aab8f5edf3ae4ba8b7066b2bbac9c56e (table test-table [id=39a47c9482fc45f1af41bacd3582d8a1]) successfully deleted
Aug 11 18:55:08 dist-test-slave-2j7r krb5kdc[31521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474508, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 18:55:08 dist-test-slave-2j7r krb5kdc[31521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474508, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM
I20260811 18:55:08.878050 32707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:08.865345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42166 (local address 127.27.180.190:40919)
0811 18:55:08.865646 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:08.865654 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:08.866395 (+ 741us) server_negotiation.cc:407] Connection header received
0811 18:55:08.866545 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:08.866548 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:08.866603 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:08.866694 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:08.867635 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:08.868402 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:08.869589 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:08.869938 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:08.870047 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:08.875359 (+ 5312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:08.875362 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:08.875364 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:08.875387 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:08.876568 (+ 1181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:08.876926 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:08.876928 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:08.876930 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:08.876998 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:08.877225 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:08.877228 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:08.877229 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:08.877401 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:08.877503 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:08.877755 (+ 252us) server_negotiation.cc:299] Negotiation successful
0811 18:55:08.877881 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":227,"thread_start_us":88,"threads_started":1}
I20260811 18:55:08.880440 31600 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42166:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:55:08.880827 31600 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:55:08.885324 31862 tablet_service.cc:1467] Processing CreateTablet for tablet 26ac8f9162224dbfaf4a12270f9c89ab (DEFAULT_TABLE table=test-table [id=b3fc5d7dde5a48f2b6fed1417c2600ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:55:08.885357 32136 tablet_service.cc:1467] Processing CreateTablet for tablet 26ac8f9162224dbfaf4a12270f9c89ab (DEFAULT_TABLE table=test-table [id=b3fc5d7dde5a48f2b6fed1417c2600ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:55:08.885447 32400 tablet_service.cc:1467] Processing CreateTablet for tablet 26ac8f9162224dbfaf4a12270f9c89ab (DEFAULT_TABLE table=test-table [id=b3fc5d7dde5a48f2b6fed1417c2600ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:55:08.885645 32136 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26ac8f9162224dbfaf4a12270f9c89ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:55:08.885638 31862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26ac8f9162224dbfaf4a12270f9c89ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:55:08.885682 32400 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26ac8f9162224dbfaf4a12270f9c89ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:55:08.888125 32715 tablet_bootstrap.cc:492] T 26ac8f9162224dbfaf4a12270f9c89ab P 55f3851f9ad643c49db10785cd71553a: Bootstrap starting.
I20260811 18:55:08.888441 32709 tablet_bootstrap.cc:492] T 26ac8f9162224dbfaf4a12270f9c89ab P 43665cdb5ae6441fbc7ffb7abed2cbaa: Bootstrap starting.
I20260811 18:55:08.889055 32715 tablet_bootstrap.cc:654] T 26ac8f9162224dbfaf4a12270f9c89ab P 55f3851f9ad643c49db10785cd71553a: Neither blocks nor log segments found. Creating new log.
I20260811 18:55:08.889320 32712 tablet_bootstrap.cc:492] T 26ac8f9162224dbfaf4a12270f9c89ab P ae1ad05a2108484483d65a2d91426f33: Bootstrap starting.
I20260811 18:55:08.889688 32709 tablet_bootstrap.cc:654] T 26ac8f9162224dbfaf4a12270f9c89ab P 43665cdb5ae6441fbc7ffb7abed2cbaa: Neither blocks nor log segments found. Creating new log.
I20260811 18:55:08.890128 32712 tablet_bootstrap.cc:654] T 26ac8f9162224dbfaf4a12270f9c89ab P ae1ad05a2108484483d65a2d91426f33: Neither blocks nor log segments found. Creating new log.
I20260811 18:55:08.890341 32715 tablet_bootstrap.cc:492] T 26ac8f9162224dbfaf4a12270f9c89ab P 55f3851f9ad643c49db10785cd71553a: No bootstrap required, opened a new log
I20260811 18:55:08.890419 32715 ts_tablet_manager.cc:1397] T 26ac8f9162224dbfaf4a12270f9c89ab P 55f3851f9ad643c49db10785cd71553a: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260811 18:55:08.890818 32715 raft_consensus.cc:348] T 26ac8f9162224dbfaf4a12270f9c89ab P 55f3851f9ad643c49db10785cd71553a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43665cdb5ae6441fbc7ffb7abed2cbaa" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 41133 } } peers { permanent_uuid: "ae1ad05a2108484483d65a2d91426f33" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 33351 } } peers { permanent_uuid: "55f3851f9ad643c49db10785cd71553a" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 35085 } }
I20260811 18:55:08.891088 32715 raft_consensus.cc:374] T 26ac8f9162224dbfaf4a12270f9c89ab P 55f3851f9ad643c49db10785cd71553a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:55:08.891146 32715 raft_consensus.cc:729] T 26ac8f9162224dbfaf4a12270f9c89ab P 55f3851f9ad643c49db10785cd71553a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55f3851f9ad643c49db10785cd71553a, State: Initialized, Role: FOLLOWER
I20260811 18:55:08.891224 32712 tablet_bootstrap.cc:492] T 26ac8f9162224dbfaf4a12270f9c89ab P ae1ad05a2108484483d65a2d91426f33: No bootstrap required, opened a new log
I20260811 18:55:08.891319 32712 ts_tablet_manager.cc:1397] T 26ac8f9162224dbfaf4a12270f9c89ab P ae1ad05a2108484483d65a2d91426f33: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 18:55:08.891325 32715 consensus_queue.cc:260] T 26ac8f9162224dbfaf4a12270f9c89ab P 55f3851f9ad643c49db10785cd71553a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43665cdb5ae6441fbc7ffb7abed2cbaa" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 41133 } } peers { permanent_uuid: "ae1ad05a2108484483d65a2d91426f33" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 33351 } } peers { permanent_uuid: "55f3851f9ad643c49db10785cd71553a" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 35085 } }
I20260811 18:55:08.891613 32715 ts_tablet_manager.cc:1428] T 26ac8f9162224dbfaf4a12270f9c89ab P 55f3851f9ad643c49db10785cd71553a: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260811 18:55:08.891686 32712 raft_consensus.cc:348] T 26ac8f9162224dbfaf4a12270f9c89ab P ae1ad05a2108484483d65a2d91426f33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43665cdb5ae6441fbc7ffb7abed2cbaa" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 41133 } } peers { permanent_uuid: "ae1ad05a2108484483d65a2d91426f33" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 33351 } } peers { permanent_uuid: "55f3851f9ad643c49db10785cd71553a" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 35085 } }
I20260811 18:55:08.891793 32712 raft_consensus.cc:374] T 26ac8f9162224dbfaf4a12270f9c89ab P ae1ad05a2108484483d65a2d91426f33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:55:08.891825 32712 raft_consensus.cc:729] T 26ac8f9162224dbfaf4a12270f9c89ab P ae1ad05a2108484483d65a2d91426f33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae1ad05a2108484483d65a2d91426f33, State: Initialized, Role: FOLLOWER
I20260811 18:55:08.891976 32712 consensus_queue.cc:260] T 26ac8f9162224dbfaf4a12270f9c89ab P ae1ad05a2108484483d65a2d91426f33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43665cdb5ae6441fbc7ffb7abed2cbaa" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 41133 } } peers { permanent_uuid: "ae1ad05a2108484483d65a2d91426f33" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 33351 } } peers { permanent_uuid: "55f3851f9ad643c49db10785cd71553a" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 35085 } }
I20260811 18:55:08.892385 32709 tablet_bootstrap.cc:492] T 26ac8f9162224dbfaf4a12270f9c89ab P 43665cdb5ae6441fbc7ffb7abed2cbaa: No bootstrap required, opened a new log
I20260811 18:55:08.892594 32712 ts_tablet_manager.cc:1428] T 26ac8f9162224dbfaf4a12270f9c89ab P ae1ad05a2108484483d65a2d91426f33: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 18:55:08.892619 32709 ts_tablet_manager.cc:1397] T 26ac8f9162224dbfaf4a12270f9c89ab P 43665cdb5ae6441fbc7ffb7abed2cbaa: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260811 18:55:08.893110 32709 raft_consensus.cc:348] T 26ac8f9162224dbfaf4a12270f9c89ab P 43665cdb5ae6441fbc7ffb7abed2cbaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43665cdb5ae6441fbc7ffb7abed2cbaa" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 41133 } } peers { permanent_uuid: "ae1ad05a2108484483d65a2d91426f33" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 33351 } } peers { permanent_uuid: "55f3851f9ad643c49db10785cd71553a" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 35085 } }
I20260811 18:55:08.893219 32709 raft_consensus.cc:374] T 26ac8f9162224dbfaf4a12270f9c89ab P 43665cdb5ae6441fbc7ffb7abed2cbaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:55:08.893252 32709 raft_consensus.cc:729] T 26ac8f9162224dbfaf4a12270f9c89ab P 43665cdb5ae6441fbc7ffb7abed2cbaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43665cdb5ae6441fbc7ffb7abed2cbaa, State: Initialized, Role: FOLLOWER
I20260811 18:55:08.893402 32709 consensus_queue.cc:260] T 26ac8f9162224dbfaf4a12270f9c89ab P 43665cdb5ae6441fbc7ffb7abed2cbaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43665cdb5ae6441fbc7ffb7abed2cbaa" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 41133 } } peers { permanent_uuid: "ae1ad05a2108484483d65a2d91426f33" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 33351 } } peers { permanent_uuid: "55f3851f9ad643c49db10785cd71553a" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 35085 } }
I20260811 18:55:08.893801 32709 ts_tablet_manager.cc:1428] T 26ac8f9162224dbfaf4a12270f9c89ab P 43665cdb5ae6441fbc7ffb7abed2cbaa: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 18:55:09.051069 32722 raft_consensus.cc:482] T 26ac8f9162224dbfaf4a12270f9c89ab P 43665cdb5ae6441fbc7ffb7abed2cbaa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:55:09.051237 32722 raft_consensus.cc:504] T 26ac8f9162224dbfaf4a12270f9c89ab P 43665cdb5ae6441fbc7ffb7abed2cbaa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43665cdb5ae6441fbc7ffb7abed2cbaa" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 41133 } } peers { permanent_uuid: "ae1ad05a2108484483d65a2d91426f33" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 33351 } } peers { permanent_uuid: "55f3851f9ad643c49db10785cd71553a" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 35085 } }
I20260811 18:55:09.052515 32722 leader_election.cc:288] T 26ac8f9162224dbfaf4a12270f9c89ab P 43665cdb5ae6441fbc7ffb7abed2cbaa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ae1ad05a2108484483d65a2d91426f33 (127.27.180.130:33351), 55f3851f9ad643c49db10785cd71553a (127.27.180.131:35085)
I20260811 18:55:09.056869 32726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:09.053373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.129:55225 (local address 127.27.180.131:35085)
0811 18:55:09.053654 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:09.053663 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:09.053677 (+ 14us) server_negotiation.cc:407] Connection header received
0811 18:55:09.053801 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:09.053804 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:09.053852 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:09.053934 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:55:09.054254 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:09.054801 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:09.055828 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:09.056247 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:09.056357 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:09.056414 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:09.056669 (+ 255us) server_negotiation.cc:299] Negotiation successful
0811 18:55:09.056757 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":205,"thread_start_us":116,"threads_started":1}
I20260811 18:55:09.056881 32725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:09.053200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.131:35085 (local address 127.27.180.129:55225)
0811 18:55:09.053591 (+ 391us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:09.053603 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:09.053685 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:09.053989 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:09.053992 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:09.054015 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:55:09.054141 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:09.054147 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:09.054934 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:09.054937 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:09.055694 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:09.055703 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:09.056363 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:09.056366 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:09.056533 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:09.056559 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 18:55:09.056630 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 18:55:09.056702 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"mutex_wait_us":55,"thread_start_us":91,"threads_started":1}
I20260811 18:55:09.056893 32723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:09.052744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.130:33351 (local address 127.27.180.129:40591)
0811 18:55:09.053180 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:09.053195 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:09.053345 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:09.053689 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:09.053692 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:09.053720 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:55:09.053835 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:09.053842 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:09.054764 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:09.054768 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:09.055671 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:09.055681 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:09.056366 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:09.056369 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:09.056531 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:09.056566 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 18:55:09.056640 (+ 74us) client_negotiation.cc:240] Negotiation successful
0811 18:55:09.056714 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":358,"mutex_wait_us":27,"thread_start_us":138,"threads_started":1}
I20260811 18:55:09.057004 32724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:09.052857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.129:40591 (local address 127.27.180.130:33351)
0811 18:55:09.053166 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:09.053176 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:09.053432 (+ 256us) server_negotiation.cc:407] Connection header received
0811 18:55:09.053499 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:09.053501 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:09.053553 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:09.053647 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:55:09.053990 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:09.054644 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:09.055820 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:09.056247 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:09.056358 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:09.056414 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:09.056798 (+ 384us) server_negotiation.cc:299] Negotiation successful
0811 18:55:09.056882 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":203,"thread_start_us":96,"threads_started":1}
I20260811 18:55:09.057463 32179 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26ac8f9162224dbfaf4a12270f9c89ab" candidate_uuid: "43665cdb5ae6441fbc7ffb7abed2cbaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae1ad05a2108484483d65a2d91426f33" is_pre_election: true
I20260811 18:55:09.057463 32421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26ac8f9162224dbfaf4a12270f9c89ab" candidate_uuid: "43665cdb5ae6441fbc7ffb7abed2cbaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55f3851f9ad643c49db10785cd71553a" is_pre_election: true
I20260811 18:55:09.057602 32421 raft_consensus.cc:2454] T 26ac8f9162224dbfaf4a12270f9c89ab P 55f3851f9ad643c49db10785cd71553a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43665cdb5ae6441fbc7ffb7abed2cbaa in term 0.
I20260811 18:55:09.057608 32179 raft_consensus.cc:2454] T 26ac8f9162224dbfaf4a12270f9c89ab P ae1ad05a2108484483d65a2d91426f33 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43665cdb5ae6441fbc7ffb7abed2cbaa in term 0.
I20260811 18:55:09.057976 31762 leader_election.cc:302] T 26ac8f9162224dbfaf4a12270f9c89ab P 43665cdb5ae6441fbc7ffb7abed2cbaa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43665cdb5ae6441fbc7ffb7abed2cbaa, ae1ad05a2108484483d65a2d91426f33; no voters:
I20260811 18:55:09.058240 32722 raft_consensus.cc:2781] T 26ac8f9162224dbfaf4a12270f9c89ab P 43665cdb5ae6441fbc7ffb7abed2cbaa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:55:09.058296 32722 raft_consensus.cc:482] T 26ac8f9162224dbfaf4a12270f9c89ab P 43665cdb5ae6441fbc7ffb7abed2cbaa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:55:09.058322 32722 raft_consensus.cc:3037] T 26ac8f9162224dbfaf4a12270f9c89ab P 43665cdb5ae6441fbc7ffb7abed2cbaa [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:55:09.059293 32722 raft_consensus.cc:504] T 26ac8f9162224dbfaf4a12270f9c89ab P 43665cdb5ae6441fbc7ffb7abed2cbaa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43665cdb5ae6441fbc7ffb7abed2cbaa" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 41133 } } peers { permanent_uuid: "ae1ad05a2108484483d65a2d91426f33" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 33351 } } peers { permanent_uuid: "55f3851f9ad643c49db10785cd71553a" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 35085 } }
I20260811 18:55:09.059607 32722 leader_election.cc:288] T 26ac8f9162224dbfaf4a12270f9c89ab P 43665cdb5ae6441fbc7ffb7abed2cbaa [CANDIDATE]: Term 1 election: Requested vote from peers ae1ad05a2108484483d65a2d91426f33 (127.27.180.130:33351), 55f3851f9ad643c49db10785cd71553a (127.27.180.131:35085)
I20260811 18:55:09.059980 32179 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26ac8f9162224dbfaf4a12270f9c89ab" candidate_uuid: "43665cdb5ae6441fbc7ffb7abed2cbaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae1ad05a2108484483d65a2d91426f33"
I20260811 18:55:09.060048 32421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26ac8f9162224dbfaf4a12270f9c89ab" candidate_uuid: "43665cdb5ae6441fbc7ffb7abed2cbaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55f3851f9ad643c49db10785cd71553a"
I20260811 18:55:09.060091 32179 raft_consensus.cc:3037] T 26ac8f9162224dbfaf4a12270f9c89ab P ae1ad05a2108484483d65a2d91426f33 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:55:09.060187 32421 raft_consensus.cc:3037] T 26ac8f9162224dbfaf4a12270f9c89ab P 55f3851f9ad643c49db10785cd71553a [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:55:09.061122 32179 raft_consensus.cc:2454] T 26ac8f9162224dbfaf4a12270f9c89ab P ae1ad05a2108484483d65a2d91426f33 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43665cdb5ae6441fbc7ffb7abed2cbaa in term 1.
I20260811 18:55:09.061267 32421 raft_consensus.cc:2454] T 26ac8f9162224dbfaf4a12270f9c89ab P 55f3851f9ad643c49db10785cd71553a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43665cdb5ae6441fbc7ffb7abed2cbaa in term 1.
I20260811 18:55:09.061385 31762 leader_election.cc:302] T 26ac8f9162224dbfaf4a12270f9c89ab P 43665cdb5ae6441fbc7ffb7abed2cbaa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43665cdb5ae6441fbc7ffb7abed2cbaa, ae1ad05a2108484483d65a2d91426f33; no voters:
I20260811 18:55:09.061556 32722 raft_consensus.cc:2781] T 26ac8f9162224dbfaf4a12270f9c89ab P 43665cdb5ae6441fbc7ffb7abed2cbaa [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:55:09.061775 32722 raft_consensus.cc:686] T 26ac8f9162224dbfaf4a12270f9c89ab P 43665cdb5ae6441fbc7ffb7abed2cbaa [term 1 LEADER]: Becoming Leader. State: Replica: 43665cdb5ae6441fbc7ffb7abed2cbaa, State: Running, Role: LEADER
I20260811 18:55:09.062000 32722 consensus_queue.cc:237] T 26ac8f9162224dbfaf4a12270f9c89ab P 43665cdb5ae6441fbc7ffb7abed2cbaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43665cdb5ae6441fbc7ffb7abed2cbaa" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 41133 } } peers { permanent_uuid: "ae1ad05a2108484483d65a2d91426f33" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 33351 } } peers { permanent_uuid: "55f3851f9ad643c49db10785cd71553a" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 35085 } }
I20260811 18:55:09.064113 31600 catalog_manager.cc:5495] T 26ac8f9162224dbfaf4a12270f9c89ab P 43665cdb5ae6441fbc7ffb7abed2cbaa reported cstate change: term changed from 0 to 1, leader changed from <none> to 43665cdb5ae6441fbc7ffb7abed2cbaa (127.27.180.129). New cstate: current_term: 1 leader_uuid: "43665cdb5ae6441fbc7ffb7abed2cbaa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43665cdb5ae6441fbc7ffb7abed2cbaa" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 41133 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ae1ad05a2108484483d65a2d91426f33" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 33351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "55f3851f9ad643c49db10785cd71553a" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 35085 } health_report { overall_health: UNKNOWN } } }
I20260811 18:55:09.084081 32733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:09.080270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51436 (local address 127.27.180.129:41133)
0811 18:55:09.080581 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:09.080611 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:09.080625 (+ 14us) server_negotiation.cc:407] Connection header received
0811 18:55:09.080689 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:09.080691 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:09.080756 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:09.080858 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:55:09.081129 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:09.081821 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:09.082658 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:09.082907 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:09.082986 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:09.083238 (+ 252us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:55:09.083704 (+ 466us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:55:09.083806 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:09.083864 (+ 58us) server_negotiation.cc:299] Negotiation successful
0811 18:55:09.083913 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":249,"thread_start_us":86,"threads_started":1}
I20260811 18:55:09.093761 32737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:09.090506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.130:33351 (local address 127.27.180.129:40679)
0811 18:55:09.090842 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:09.090854 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:09.090938 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:09.091120 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:09.091122 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:09.091127 (+ 5us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 18:55:09.091217 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:09.091221 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:09.091874 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:09.091876 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:09.092562 (+ 686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:09.092571 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:09.093021 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:09.093025 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:09.093155 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:09.093202 (+ 47us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 18:55:09.093482 (+ 280us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 18:55:09.093488 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 18:55:09.093545 (+ 57us) client_negotiation.cc:240] Negotiation successful
0811 18:55:09.093622 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":101,"threads_started":1}
I20260811 18:55:09.093747 32724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:09.090714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.129:40679 (local address 127.27.180.130:33351)
0811 18:55:09.090821 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:09.090824 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:09.090913 (+ 89us) server_negotiation.cc:407] Connection header received
0811 18:55:09.091008 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:09.091010 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:09.091050 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:09.091110 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:55:09.091326 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:09.091801 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:09.092672 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:09.092910 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:09.093000 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:09.093337 (+ 337us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:55:09.093403 (+ 66us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:55:09.093461 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:09.093556 (+ 95us) server_negotiation.cc:299] Negotiation successful
0811 18:55:09.093607 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 18:55:09.098990 32421 raft_consensus.cc:1261] T 26ac8f9162224dbfaf4a12270f9c89ab P 55f3851f9ad643c49db10785cd71553a [term 1 FOLLOWER]: Refusing update from remote peer 43665cdb5ae6441fbc7ffb7abed2cbaa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:55:09.099061 32179 raft_consensus.cc:1261] T 26ac8f9162224dbfaf4a12270f9c89ab P ae1ad05a2108484483d65a2d91426f33 [term 1 FOLLOWER]: Refusing update from remote peer 43665cdb5ae6441fbc7ffb7abed2cbaa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:55:09.099416 32727 consensus_queue.cc:1035] T 26ac8f9162224dbfaf4a12270f9c89ab P 43665cdb5ae6441fbc7ffb7abed2cbaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "55f3851f9ad643c49db10785cd71553a" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 35085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:55:09.099582 32722 consensus_queue.cc:1035] T 26ac8f9162224dbfaf4a12270f9c89ab P 43665cdb5ae6441fbc7ffb7abed2cbaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ae1ad05a2108484483d65a2d91426f33" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 33351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:55:09.116798 32733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:09.113520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.130:38409 (local address 127.27.180.129:41133)
0811 18:55:09.113625 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:09.113629 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:09.113886 (+ 257us) server_negotiation.cc:407] Connection header received
0811 18:55:09.113978 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:09.113980 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:09.114020 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:09.114074 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:55:09.114217 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:09.114722 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:09.115720 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:09.116014 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:09.116117 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:09.116402 (+ 285us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:55:09.116478 (+ 76us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:55:09.116544 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:09.116645 (+ 101us) server_negotiation.cc:299] Negotiation successful
0811 18:55:09.116697 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 18:55:09.116833 32743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:09.113420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.129:41133 (local address 127.27.180.130:38409)
0811 18:55:09.113799 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:09.113814 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:09.113915 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:09.114078 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:09.114080 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:09.114085 (+ 5us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 18:55:09.114163 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:09.114167 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:09.114800 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:09.114802 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:09.115594 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:09.115604 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:09.116130 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:09.116134 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:09.116216 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:09.116281 (+ 65us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 18:55:09.116560 (+ 279us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 18:55:09.116567 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 18:55:09.116625 (+ 58us) client_negotiation.cc:240] Negotiation successful
0811 18:55:09.116709 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":117,"threads_started":1}
I20260811 18:55:09.137650 31600 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42166:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:55:09.137786 31600 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42166:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:55:09.139690 31600 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3b87a006e181434094f7d5896eb20643: Sending DeleteTablet for 3 replicas of tablet 26ac8f9162224dbfaf4a12270f9c89ab
I20260811 18:55:09.140276 32400 tablet_service.cc:1514] Processing DeleteTablet for tablet 26ac8f9162224dbfaf4a12270f9c89ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:55:09 UTC) from {username='kudu', principal='kudu/127.27.180.190@KRBTEST.COM'} at 127.0.0.1:48190
I20260811 18:55:09.140403 32136 tablet_service.cc:1514] Processing DeleteTablet for tablet 26ac8f9162224dbfaf4a12270f9c89ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:55:09 UTC) from {username='kudu', principal='kudu/127.27.180.190@KRBTEST.COM'} at 127.0.0.1:52818
I20260811 18:55:09.140451 32693 tablet_replica.cc:330] T 26ac8f9162224dbfaf4a12270f9c89ab P 55f3851f9ad643c49db10785cd71553a: stopping tablet replica
I20260811 18:55:09.140556 32693 raft_consensus.cc:2229] T 26ac8f9162224dbfaf4a12270f9c89ab P 55f3851f9ad643c49db10785cd71553a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:55:09.140578 32692 tablet_replica.cc:330] T 26ac8f9162224dbfaf4a12270f9c89ab P ae1ad05a2108484483d65a2d91426f33: stopping tablet replica
I20260811 18:55:09.140626 31862 tablet_service.cc:1514] Processing DeleteTablet for tablet 26ac8f9162224dbfaf4a12270f9c89ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:55:09 UTC) from {username='kudu', principal='kudu/127.27.180.190@KRBTEST.COM'} at 127.0.0.1:51424
I20260811 18:55:09.140659 32693 raft_consensus.cc:2258] T 26ac8f9162224dbfaf4a12270f9c89ab P 55f3851f9ad643c49db10785cd71553a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:55:09.140708 32692 raft_consensus.cc:2229] T 26ac8f9162224dbfaf4a12270f9c89ab P ae1ad05a2108484483d65a2d91426f33 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:55:09.140823 32692 raft_consensus.cc:2258] T 26ac8f9162224dbfaf4a12270f9c89ab P ae1ad05a2108484483d65a2d91426f33 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:55:09.140826 32691 tablet_replica.cc:330] T 26ac8f9162224dbfaf4a12270f9c89ab P 43665cdb5ae6441fbc7ffb7abed2cbaa: stopping tablet replica
I20260811 18:55:09.140950 32691 raft_consensus.cc:2229] T 26ac8f9162224dbfaf4a12270f9c89ab P 43665cdb5ae6441fbc7ffb7abed2cbaa [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:55:09.141242 32691 raft_consensus.cc:2258] T 26ac8f9162224dbfaf4a12270f9c89ab P 43665cdb5ae6441fbc7ffb7abed2cbaa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:55:09.141358 31600 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42166:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:55:09.141518 32693 ts_tablet_manager.cc:1905] T 26ac8f9162224dbfaf4a12270f9c89ab P 55f3851f9ad643c49db10785cd71553a: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:55:09.141749 31600 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:55:09.141995 32691 ts_tablet_manager.cc:1905] T 26ac8f9162224dbfaf4a12270f9c89ab P 43665cdb5ae6441fbc7ffb7abed2cbaa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:55:09.144076 32693 ts_tablet_manager.cc:1918] T 26ac8f9162224dbfaf4a12270f9c89ab P 55f3851f9ad643c49db10785cd71553a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 18:55:09.144164 32693 log.cc:1196] T 26ac8f9162224dbfaf4a12270f9c89ab P 55f3851f9ad643c49db10785cd71553a: Deleting WAL directory at /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/minicluster-data/ts-2/wal/wals/26ac8f9162224dbfaf4a12270f9c89ab
I20260811 18:55:09.144363 32691 ts_tablet_manager.cc:1918] T 26ac8f9162224dbfaf4a12270f9c89ab P 43665cdb5ae6441fbc7ffb7abed2cbaa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 18:55:09.144438 32691 log.cc:1196] T 26ac8f9162224dbfaf4a12270f9c89ab P 43665cdb5ae6441fbc7ffb7abed2cbaa: Deleting WAL directory at /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/minicluster-data/ts-0/wal/wals/26ac8f9162224dbfaf4a12270f9c89ab
I20260811 18:55:09.144658 32691 ts_tablet_manager.cc:1939] T 26ac8f9162224dbfaf4a12270f9c89ab P 43665cdb5ae6441fbc7ffb7abed2cbaa: Deleting consensus metadata
I20260811 18:55:09.144831 32692 ts_tablet_manager.cc:1905] T 26ac8f9162224dbfaf4a12270f9c89ab P ae1ad05a2108484483d65a2d91426f33: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:55:09.145395 31568 catalog_manager.cc:4841] TS 43665cdb5ae6441fbc7ffb7abed2cbaa (127.27.180.129:41133): tablet 26ac8f9162224dbfaf4a12270f9c89ab (table test-table [id=b3fc5d7dde5a48f2b6fed1417c2600ec]) successfully deleted
I20260811 18:55:09.146169 32693 ts_tablet_manager.cc:1939] T 26ac8f9162224dbfaf4a12270f9c89ab P 55f3851f9ad643c49db10785cd71553a: Deleting consensus metadata
I20260811 18:55:09.146884 31574 catalog_manager.cc:4841] TS 55f3851f9ad643c49db10785cd71553a (127.27.180.131:35085): tablet 26ac8f9162224dbfaf4a12270f9c89ab (table test-table [id=b3fc5d7dde5a48f2b6fed1417c2600ec]) successfully deleted
I20260811 18:55:09.147333 32692 ts_tablet_manager.cc:1918] T 26ac8f9162224dbfaf4a12270f9c89ab P ae1ad05a2108484483d65a2d91426f33: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 18:55:09.147416 32692 log.cc:1196] T 26ac8f9162224dbfaf4a12270f9c89ab P ae1ad05a2108484483d65a2d91426f33: Deleting WAL directory at /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/minicluster-data/ts-1/wal/wals/26ac8f9162224dbfaf4a12270f9c89ab
I20260811 18:55:09.147658 32692 ts_tablet_manager.cc:1939] T 26ac8f9162224dbfaf4a12270f9c89ab P ae1ad05a2108484483d65a2d91426f33: Deleting consensus metadata
I20260811 18:55:09.148267 31574 catalog_manager.cc:4841] TS ae1ad05a2108484483d65a2d91426f33 (127.27.180.130:33351): tablet 26ac8f9162224dbfaf4a12270f9c89ab (table test-table [id=b3fc5d7dde5a48f2b6fed1417c2600ec]) successfully deleted
I20260811 18:55:09.149045 31862 tablet_service.cc:1467] Processing CreateTablet for tablet dc83374f1e8b45348b15ac9d94de8cfc (DEFAULT_TABLE table=test-table [id=62d260cb5e924b3a982a48fd193cd3af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:55:09.149315 31862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc83374f1e8b45348b15ac9d94de8cfc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:55:09.149257 32136 tablet_service.cc:1467] Processing CreateTablet for tablet dc83374f1e8b45348b15ac9d94de8cfc (DEFAULT_TABLE table=test-table [id=62d260cb5e924b3a982a48fd193cd3af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:55:09.149345 32400 tablet_service.cc:1467] Processing CreateTablet for tablet dc83374f1e8b45348b15ac9d94de8cfc (DEFAULT_TABLE table=test-table [id=62d260cb5e924b3a982a48fd193cd3af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:55:09.149473 32136 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc83374f1e8b45348b15ac9d94de8cfc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:55:09.149582 32400 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc83374f1e8b45348b15ac9d94de8cfc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:55:09.151469 32712 tablet_bootstrap.cc:492] T dc83374f1e8b45348b15ac9d94de8cfc P ae1ad05a2108484483d65a2d91426f33: Bootstrap starting.
I20260811 18:55:09.151655 32715 tablet_bootstrap.cc:492] T dc83374f1e8b45348b15ac9d94de8cfc P 55f3851f9ad643c49db10785cd71553a: Bootstrap starting.
I20260811 18:55:09.151717 32709 tablet_bootstrap.cc:492] T dc83374f1e8b45348b15ac9d94de8cfc P 43665cdb5ae6441fbc7ffb7abed2cbaa: Bootstrap starting.
I20260811 18:55:09.152243 32712 tablet_bootstrap.cc:654] T dc83374f1e8b45348b15ac9d94de8cfc P ae1ad05a2108484483d65a2d91426f33: Neither blocks nor log segments found. Creating new log.
I20260811 18:55:09.152356 32715 tablet_bootstrap.cc:654] T dc83374f1e8b45348b15ac9d94de8cfc P 55f3851f9ad643c49db10785cd71553a: Neither blocks nor log segments found. Creating new log.
I20260811 18:55:09.152711 32709 tablet_bootstrap.cc:654] T dc83374f1e8b45348b15ac9d94de8cfc P 43665cdb5ae6441fbc7ffb7abed2cbaa: Neither blocks nor log segments found. Creating new log.
I20260811 18:55:09.153424 32712 tablet_bootstrap.cc:492] T dc83374f1e8b45348b15ac9d94de8cfc P ae1ad05a2108484483d65a2d91426f33: No bootstrap required, opened a new log
I20260811 18:55:09.153510 32712 ts_tablet_manager.cc:1397] T dc83374f1e8b45348b15ac9d94de8cfc P ae1ad05a2108484483d65a2d91426f33: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260811 18:55:09.153885 32712 raft_consensus.cc:348] T dc83374f1e8b45348b15ac9d94de8cfc P ae1ad05a2108484483d65a2d91426f33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43665cdb5ae6441fbc7ffb7abed2cbaa" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 41133 } } peers { permanent_uuid: "ae1ad05a2108484483d65a2d91426f33" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 33351 } } peers { permanent_uuid: "55f3851f9ad643c49db10785cd71553a" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 35085 } }
I20260811 18:55:09.153995 32712 raft_consensus.cc:374] T dc83374f1e8b45348b15ac9d94de8cfc P ae1ad05a2108484483d65a2d91426f33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:55:09.154028 32712 raft_consensus.cc:729] T dc83374f1e8b45348b15ac9d94de8cfc P ae1ad05a2108484483d65a2d91426f33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae1ad05a2108484483d65a2d91426f33, State: Initialized, Role: FOLLOWER
I20260811 18:55:09.154142 32712 consensus_queue.cc:260] T dc83374f1e8b45348b15ac9d94de8cfc P ae1ad05a2108484483d65a2d91426f33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43665cdb5ae6441fbc7ffb7abed2cbaa" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 41133 } } peers { permanent_uuid: "ae1ad05a2108484483d65a2d91426f33" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 33351 } } peers { permanent_uuid: "55f3851f9ad643c49db10785cd71553a" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 35085 } }
I20260811 18:55:09.154381 32709 tablet_bootstrap.cc:492] T dc83374f1e8b45348b15ac9d94de8cfc P 43665cdb5ae6441fbc7ffb7abed2cbaa: No bootstrap required, opened a new log
I20260811 18:55:09.154395 32712 ts_tablet_manager.cc:1428] T dc83374f1e8b45348b15ac9d94de8cfc P ae1ad05a2108484483d65a2d91426f33: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260811 18:55:09.154458 32709 ts_tablet_manager.cc:1397] T dc83374f1e8b45348b15ac9d94de8cfc P 43665cdb5ae6441fbc7ffb7abed2cbaa: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260811 18:55:09.154637 32715 tablet_bootstrap.cc:492] T dc83374f1e8b45348b15ac9d94de8cfc P 55f3851f9ad643c49db10785cd71553a: No bootstrap required, opened a new log
I20260811 18:55:09.154717 32715 ts_tablet_manager.cc:1397] T dc83374f1e8b45348b15ac9d94de8cfc P 55f3851f9ad643c49db10785cd71553a: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260811 18:55:09.154909 32709 raft_consensus.cc:348] T dc83374f1e8b45348b15ac9d94de8cfc P 43665cdb5ae6441fbc7ffb7abed2cbaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43665cdb5ae6441fbc7ffb7abed2cbaa" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 41133 } } peers { permanent_uuid: "ae1ad05a2108484483d65a2d91426f33" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 33351 } } peers { permanent_uuid: "55f3851f9ad643c49db10785cd71553a" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 35085 } }
I20260811 18:55:09.155014 32709 raft_consensus.cc:374] T dc83374f1e8b45348b15ac9d94de8cfc P 43665cdb5ae6441fbc7ffb7abed2cbaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:55:09.155046 32709 raft_consensus.cc:729] T dc83374f1e8b45348b15ac9d94de8cfc P 43665cdb5ae6441fbc7ffb7abed2cbaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43665cdb5ae6441fbc7ffb7abed2cbaa, State: Initialized, Role: FOLLOWER
I20260811 18:55:09.155107 32715 raft_consensus.cc:348] T dc83374f1e8b45348b15ac9d94de8cfc P 55f3851f9ad643c49db10785cd71553a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43665cdb5ae6441fbc7ffb7abed2cbaa" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 41133 } } peers { permanent_uuid: "ae1ad05a2108484483d65a2d91426f33" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 33351 } } peers { permanent_uuid: "55f3851f9ad643c49db10785cd71553a" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 35085 } }
I20260811 18:55:09.155197 32715 raft_consensus.cc:374] T dc83374f1e8b45348b15ac9d94de8cfc P 55f3851f9ad643c49db10785cd71553a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:55:09.155228 32715 raft_consensus.cc:729] T dc83374f1e8b45348b15ac9d94de8cfc P 55f3851f9ad643c49db10785cd71553a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55f3851f9ad643c49db10785cd71553a, State: Initialized, Role: FOLLOWER
I20260811 18:55:09.155189 32709 consensus_queue.cc:260] T dc83374f1e8b45348b15ac9d94de8cfc P 43665cdb5ae6441fbc7ffb7abed2cbaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43665cdb5ae6441fbc7ffb7abed2cbaa" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 41133 } } peers { permanent_uuid: "ae1ad05a2108484483d65a2d91426f33" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 33351 } } peers { permanent_uuid: "55f3851f9ad643c49db10785cd71553a" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 35085 } }
I20260811 18:55:09.155339 32715 consensus_queue.cc:260] T dc83374f1e8b45348b15ac9d94de8cfc P 55f3851f9ad643c49db10785cd71553a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43665cdb5ae6441fbc7ffb7abed2cbaa" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 41133 } } peers { permanent_uuid: "ae1ad05a2108484483d65a2d91426f33" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 33351 } } peers { permanent_uuid: "55f3851f9ad643c49db10785cd71553a" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 35085 } }
I20260811 18:55:09.155475 32709 ts_tablet_manager.cc:1428] T dc83374f1e8b45348b15ac9d94de8cfc P 43665cdb5ae6441fbc7ffb7abed2cbaa: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260811 18:55:09.155586 32715 ts_tablet_manager.cc:1428] T dc83374f1e8b45348b15ac9d94de8cfc P 55f3851f9ad643c49db10785cd71553a: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 18:55:09.240760 32683 raft_consensus.cc:482] T dc83374f1e8b45348b15ac9d94de8cfc P 55f3851f9ad643c49db10785cd71553a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:55:09.240911 32683 raft_consensus.cc:504] T dc83374f1e8b45348b15ac9d94de8cfc P 55f3851f9ad643c49db10785cd71553a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43665cdb5ae6441fbc7ffb7abed2cbaa" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 41133 } } peers { permanent_uuid: "ae1ad05a2108484483d65a2d91426f33" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 33351 } } peers { permanent_uuid: "55f3851f9ad643c49db10785cd71553a" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 35085 } }
I20260811 18:55:09.241411 32683 leader_election.cc:288] T dc83374f1e8b45348b15ac9d94de8cfc P 55f3851f9ad643c49db10785cd71553a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43665cdb5ae6441fbc7ffb7abed2cbaa (127.27.180.129:41133), ae1ad05a2108484483d65a2d91426f33 (127.27.180.130:33351)
I20260811 18:55:09.241886 31908 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc83374f1e8b45348b15ac9d94de8cfc" candidate_uuid: "55f3851f9ad643c49db10785cd71553a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43665cdb5ae6441fbc7ffb7abed2cbaa" is_pre_election: true
I20260811 18:55:09.241886 32179 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc83374f1e8b45348b15ac9d94de8cfc" candidate_uuid: "55f3851f9ad643c49db10785cd71553a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae1ad05a2108484483d65a2d91426f33" is_pre_election: true
I20260811 18:55:09.242045 32179 raft_consensus.cc:2454] T dc83374f1e8b45348b15ac9d94de8cfc P ae1ad05a2108484483d65a2d91426f33 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 55f3851f9ad643c49db10785cd71553a in term 0.
I20260811 18:55:09.242045 31908 raft_consensus.cc:2454] T dc83374f1e8b45348b15ac9d94de8cfc P 43665cdb5ae6441fbc7ffb7abed2cbaa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 55f3851f9ad643c49db10785cd71553a in term 0.
I20260811 18:55:09.242375 32308 leader_election.cc:302] T dc83374f1e8b45348b15ac9d94de8cfc P 55f3851f9ad643c49db10785cd71553a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55f3851f9ad643c49db10785cd71553a, ae1ad05a2108484483d65a2d91426f33; no voters:
I20260811 18:55:09.242633 32683 raft_consensus.cc:2781] T dc83374f1e8b45348b15ac9d94de8cfc P 55f3851f9ad643c49db10785cd71553a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:55:09.242686 32683 raft_consensus.cc:482] T dc83374f1e8b45348b15ac9d94de8cfc P 55f3851f9ad643c49db10785cd71553a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:55:09.242712 32683 raft_consensus.cc:3037] T dc83374f1e8b45348b15ac9d94de8cfc P 55f3851f9ad643c49db10785cd71553a [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:55:09.243728 32683 raft_consensus.cc:504] T dc83374f1e8b45348b15ac9d94de8cfc P 55f3851f9ad643c49db10785cd71553a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43665cdb5ae6441fbc7ffb7abed2cbaa" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 41133 } } peers { permanent_uuid: "ae1ad05a2108484483d65a2d91426f33" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 33351 } } peers { permanent_uuid: "55f3851f9ad643c49db10785cd71553a" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 35085 } }
I20260811 18:55:09.244065 32683 leader_election.cc:288] T dc83374f1e8b45348b15ac9d94de8cfc P 55f3851f9ad643c49db10785cd71553a [CANDIDATE]: Term 1 election: Requested vote from peers 43665cdb5ae6441fbc7ffb7abed2cbaa (127.27.180.129:41133), ae1ad05a2108484483d65a2d91426f33 (127.27.180.130:33351)
I20260811 18:55:09.244436 32179 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc83374f1e8b45348b15ac9d94de8cfc" candidate_uuid: "55f3851f9ad643c49db10785cd71553a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae1ad05a2108484483d65a2d91426f33"
I20260811 18:55:09.244439 31908 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc83374f1e8b45348b15ac9d94de8cfc" candidate_uuid: "55f3851f9ad643c49db10785cd71553a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43665cdb5ae6441fbc7ffb7abed2cbaa"
I20260811 18:55:09.244566 32179 raft_consensus.cc:3037] T dc83374f1e8b45348b15ac9d94de8cfc P ae1ad05a2108484483d65a2d91426f33 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:55:09.244590 31908 raft_consensus.cc:3037] T dc83374f1e8b45348b15ac9d94de8cfc P 43665cdb5ae6441fbc7ffb7abed2cbaa [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:55:09.245553 32179 raft_consensus.cc:2454] T dc83374f1e8b45348b15ac9d94de8cfc P ae1ad05a2108484483d65a2d91426f33 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55f3851f9ad643c49db10785cd71553a in term 1.
I20260811 18:55:09.245833 32308 leader_election.cc:302] T dc83374f1e8b45348b15ac9d94de8cfc P 55f3851f9ad643c49db10785cd71553a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55f3851f9ad643c49db10785cd71553a, ae1ad05a2108484483d65a2d91426f33; no voters:
I20260811 18:55:09.245913 31908 raft_consensus.cc:2454] T dc83374f1e8b45348b15ac9d94de8cfc P 43665cdb5ae6441fbc7ffb7abed2cbaa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55f3851f9ad643c49db10785cd71553a in term 1.
I20260811 18:55:09.245997 32683 raft_consensus.cc:2781] T dc83374f1e8b45348b15ac9d94de8cfc P 55f3851f9ad643c49db10785cd71553a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:55:09.246073 32683 raft_consensus.cc:686] T dc83374f1e8b45348b15ac9d94de8cfc P 55f3851f9ad643c49db10785cd71553a [term 1 LEADER]: Becoming Leader. State: Replica: 55f3851f9ad643c49db10785cd71553a, State: Running, Role: LEADER
I20260811 18:55:09.246204 32683 consensus_queue.cc:237] T dc83374f1e8b45348b15ac9d94de8cfc P 55f3851f9ad643c49db10785cd71553a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43665cdb5ae6441fbc7ffb7abed2cbaa" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 41133 } } peers { permanent_uuid: "ae1ad05a2108484483d65a2d91426f33" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 33351 } } peers { permanent_uuid: "55f3851f9ad643c49db10785cd71553a" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 35085 } }
I20260811 18:55:09.247946 31599 catalog_manager.cc:5495] T dc83374f1e8b45348b15ac9d94de8cfc P 55f3851f9ad643c49db10785cd71553a reported cstate change: term changed from 0 to 1, leader changed from <none> to 55f3851f9ad643c49db10785cd71553a (127.27.180.131). New cstate: current_term: 1 leader_uuid: "55f3851f9ad643c49db10785cd71553a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43665cdb5ae6441fbc7ffb7abed2cbaa" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 41133 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ae1ad05a2108484483d65a2d91426f33" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 33351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "55f3851f9ad643c49db10785cd71553a" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 35085 } health_report { overall_health: HEALTHY } } }
I20260811 18:55:09.263069 32726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:09.259925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48206 (local address 127.27.180.131:35085)
0811 18:55:09.260035 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:09.260039 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:09.260149 (+ 110us) server_negotiation.cc:407] Connection header received
0811 18:55:09.260281 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:09.260284 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:09.260339 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:09.260428 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:55:09.260653 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:09.261254 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:09.262100 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:09.262326 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:09.262420 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:09.262649 (+ 229us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:55:09.262724 (+ 75us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:55:09.262781 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:09.262871 (+ 90us) server_negotiation.cc:299] Negotiation successful
0811 18:55:09.262929 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260811 18:55:09.269891 31907 raft_consensus.cc:1261] T dc83374f1e8b45348b15ac9d94de8cfc P 43665cdb5ae6441fbc7ffb7abed2cbaa [term 1 FOLLOWER]: Refusing update from remote peer 55f3851f9ad643c49db10785cd71553a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:55:09.269876 32179 raft_consensus.cc:1261] T dc83374f1e8b45348b15ac9d94de8cfc P ae1ad05a2108484483d65a2d91426f33 [term 1 FOLLOWER]: Refusing update from remote peer 55f3851f9ad643c49db10785cd71553a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:55:09.270335 32675 consensus_queue.cc:1035] T dc83374f1e8b45348b15ac9d94de8cfc P 55f3851f9ad643c49db10785cd71553a [LEADER]: Connected to new peer: Peer: permanent_uuid: "43665cdb5ae6441fbc7ffb7abed2cbaa" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 41133 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:55:09.270618 32675 consensus_queue.cc:1035] T dc83374f1e8b45348b15ac9d94de8cfc P 55f3851f9ad643c49db10785cd71553a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ae1ad05a2108484483d65a2d91426f33" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 33351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 18:55:09.286906 32659 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260811 18:55:10.838209 31592 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42166:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:55:10.838372 31592 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42166:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:55:10.840761 31592 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3b87a006e181434094f7d5896eb20643: Sending DeleteTablet for 3 replicas of tablet dc83374f1e8b45348b15ac9d94de8cfc
I20260811 18:55:10.841290 31862 tablet_service.cc:1514] Processing DeleteTablet for tablet dc83374f1e8b45348b15ac9d94de8cfc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:55:10 UTC) from {username='kudu', principal='kudu/127.27.180.190@KRBTEST.COM'} at 127.0.0.1:51424
I20260811 18:55:10.841331 32136 tablet_service.cc:1514] Processing DeleteTablet for tablet dc83374f1e8b45348b15ac9d94de8cfc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:55:10 UTC) from {username='kudu', principal='kudu/127.27.180.190@KRBTEST.COM'} at 127.0.0.1:52818
I20260811 18:55:10.841428 32400 tablet_service.cc:1514] Processing DeleteTablet for tablet dc83374f1e8b45348b15ac9d94de8cfc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:55:10 UTC) from {username='kudu', principal='kudu/127.27.180.190@KRBTEST.COM'} at 127.0.0.1:48190
I20260811 18:55:10.841652 341 tablet_replica.cc:330] T dc83374f1e8b45348b15ac9d94de8cfc P ae1ad05a2108484483d65a2d91426f33: stopping tablet replica
I20260811 18:55:10.841755 345 tablet_replica.cc:330] T dc83374f1e8b45348b15ac9d94de8cfc P 55f3851f9ad643c49db10785cd71553a: stopping tablet replica
I20260811 18:55:10.841796 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 31732
I20260811 18:55:10.861930 341 raft_consensus.cc:2229] T dc83374f1e8b45348b15ac9d94de8cfc P ae1ad05a2108484483d65a2d91426f33 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:55:10.862262 341 raft_consensus.cc:2258] T dc83374f1e8b45348b15ac9d94de8cfc P ae1ad05a2108484483d65a2d91426f33 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:55:10.863178 341 ts_tablet_manager.cc:1905] T dc83374f1e8b45348b15ac9d94de8cfc P ae1ad05a2108484483d65a2d91426f33: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:55:10.865389 345 raft_consensus.cc:2229] T dc83374f1e8b45348b15ac9d94de8cfc P 55f3851f9ad643c49db10785cd71553a [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:55:10.865712 345 raft_consensus.cc:2258] T dc83374f1e8b45348b15ac9d94de8cfc P 55f3851f9ad643c49db10785cd71553a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:55:10.867274 341 ts_tablet_manager.cc:1918] T dc83374f1e8b45348b15ac9d94de8cfc P ae1ad05a2108484483d65a2d91426f33: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 18:55:10.867487 341 log.cc:1196] T dc83374f1e8b45348b15ac9d94de8cfc P ae1ad05a2108484483d65a2d91426f33: Deleting WAL directory at /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/minicluster-data/ts-1/wal/wals/dc83374f1e8b45348b15ac9d94de8cfc
I20260811 18:55:10.867909 341 ts_tablet_manager.cc:1939] T dc83374f1e8b45348b15ac9d94de8cfc P ae1ad05a2108484483d65a2d91426f33: Deleting consensus metadata
I20260811 18:55:10.868221 345 ts_tablet_manager.cc:1905] T dc83374f1e8b45348b15ac9d94de8cfc P 55f3851f9ad643c49db10785cd71553a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:55:10.870754 345 ts_tablet_manager.cc:1918] T dc83374f1e8b45348b15ac9d94de8cfc P 55f3851f9ad643c49db10785cd71553a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 18:55:10.870836 345 log.cc:1196] T dc83374f1e8b45348b15ac9d94de8cfc P 55f3851f9ad643c49db10785cd71553a: Deleting WAL directory at /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786474496157106-28370-0/minicluster-data/ts-2/wal/wals/dc83374f1e8b45348b15ac9d94de8cfc
I20260811 18:55:10.871098 345 ts_tablet_manager.cc:1939] T dc83374f1e8b45348b15ac9d94de8cfc P 55f3851f9ad643c49db10785cd71553a: Deleting consensus metadata
I20260811 18:55:10.876024 31574 catalog_manager.cc:4841] TS 55f3851f9ad643c49db10785cd71553a (127.27.180.131:35085): tablet dc83374f1e8b45348b15ac9d94de8cfc (table test-table [id=62d260cb5e924b3a982a48fd193cd3af]) successfully deleted
I20260811 18:55:10.876221 31574 catalog_manager.cc:4841] TS ae1ad05a2108484483d65a2d91426f33 (127.27.180.130:33351): tablet dc83374f1e8b45348b15ac9d94de8cfc (table test-table [id=62d260cb5e924b3a982a48fd193cd3af]) successfully deleted
W20260811 18:55:10.884315 31568 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.27.180.129:41133 (error 108)
I20260811 18:55:10.884454 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 32002
I20260811 18:55:10.897243 349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:10.893883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.129:41133 (local address 127.0.0.1:51450)
0811 18:55:10.896989 (+ 3106us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:10.897110 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.129:41133: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3003,"thread_start_us":1163,"threads_started":1}
W20260811 18:55:10.897430 31568 catalog_manager.cc:4584] TS 43665cdb5ae6441fbc7ffb7abed2cbaa (127.27.180.129:41133): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet dc83374f1e8b45348b15ac9d94de8cfc: Network error: Client connection negotiation failed: client connection to 127.27.180.129:41133: connect: Connection refused (error 111)
I20260811 18:55:10.903945 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 32278
I20260811 18:55:10.919003 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 31543
2026-08-11T18:55:10Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6385 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474496157106-28370-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:55:10 dist-test-slave-2j7r krb5kdc[354](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:55:10 dist-test-slave-2j7r krb5kdc[354](info): set up 2 sockets
Aug 11 18:55:10 dist-test-slave-2j7r krb5kdc[354](info): commencing operation
krb5kdc: starting...
W20260811 18:55:12.115592 28370 mini_kdc.cc:121] Time spent starting KDC: real 1.164s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474496157106-28370-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474496157106-28370-0/krb5kdc/test-user.keytab.
Aug 11 18:55:12 dist-test-slave-2j7r krb5kdc[354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474512, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:55:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:55:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.180.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.180.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.180.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab.
Entry for principal kudu/127.27.180.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab.
I20260811 18:55:12.232342 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474496157106-28370-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474496157106-28370-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474496157106-28370-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.190:45397
--webserver_interface=127.27.180.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab
--principal=kudu/127.27.180.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:39419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.180.190:45397
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:55:12.340349 383 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:55:12.340646 383 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:55:12.340687 383 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:55:12.344799 383 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:55:12.344873 383 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:55:12.344892 383 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:55:12.344907 383 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:55:12.344921 383 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:55:12.350481 383 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:39419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474496157106-28370-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474496157106-28370-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.180.190:45397
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab
--principal=kudu/127.27.180.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.190:45397
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.180.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.383
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474496157106-28370-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:55:12.351608 383 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:55:12.352686 383 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 18:55:12.361440 383 server_base.cc:1034] running on GCE node
W20260811 18:55:12.361402 394 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:12.361402 396 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:12.361408 402 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:55:12.362207 383 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:55:12.363301 383 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:55:12.364482 383 hybrid_clock.cc:648] HybridClock initialized: now 1786474512364457 us; error 36 us; skew 500 ppm
Aug 11 18:55:12 dist-test-slave-2j7r krb5kdc[354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474512, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:55:12.367326 383 init.cc:377] Logged in from keytab as kudu/127.27.180.190@KRBTEST.COM (short username kudu)
I20260811 18:55:12.368254 383 webserver.cc:460] Webserver started at http://127.27.180.190:38089/ using document root <none> and password file <none>
I20260811 18:55:12.368800 383 fs_manager.cc:359] Metadata directory not provided
I20260811 18:55:12.368870 383 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:55:12.369078 383 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:55:12.371243 383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474496157106-28370-0/minicluster-data/master-0/data/instance:
uuid: "cddb4b6c9c5d4f8f8160335cc9dc22fd"
format_stamp: "Formatted at 2026-08-11 18:55:12 on dist-test-slave-2j7r"
server_key: "d1024726f2c5774e0f30c369a8baaf82"
server_key_iv: "2a5b7b4c05a386ec8a99583dcd725d23"
server_key_version: "encryptionkey@0"
I20260811 18:55:12.371785 383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474496157106-28370-0/minicluster-data/master-0/wal/instance:
uuid: "cddb4b6c9c5d4f8f8160335cc9dc22fd"
format_stamp: "Formatted at 2026-08-11 18:55:12 on dist-test-slave-2j7r"
server_key: "d1024726f2c5774e0f30c369a8baaf82"
server_key_iv: "2a5b7b4c05a386ec8a99583dcd725d23"
server_key_version: "encryptionkey@0"
I20260811 18:55:12.375563 383 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 18:55:12.377926 412 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:12.379106 383 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 18:55:12.379242 383 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474496157106-28370-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474496157106-28370-0/minicluster-data/master-0/wal
uuid: "cddb4b6c9c5d4f8f8160335cc9dc22fd"
format_stamp: "Formatted at 2026-08-11 18:55:12 on dist-test-slave-2j7r"
server_key: "d1024726f2c5774e0f30c369a8baaf82"
server_key_iv: "2a5b7b4c05a386ec8a99583dcd725d23"
server_key_version: "encryptionkey@0"
I20260811 18:55:12.379339 383 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474496157106-28370-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474496157106-28370-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474496157106-28370-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:55:12.395898 383 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:55:12.398761 383 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:55:12.398934 383 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:55:12.405815 383 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.190:45397
I20260811 18:55:12.405840 506 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.190:45397 every 8 connection(s)
I20260811 18:55:12.406744 383 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
I20260811 18:55:12.408084 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 383
I20260811 18:55:12.408172 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474496157106-28370-0/minicluster-data/master-0/wal/instance
I20260811 18:55:12.408439 28370 external_mini_cluster.cc:1428] Setting key fb286d0cd8ef5d64251ae943829085a8
I20260811 18:55:12.409706 507 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:55:12.416486 507 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cddb4b6c9c5d4f8f8160335cc9dc22fd: Bootstrap starting.
Aug 11 18:55:12 dist-test-slave-2j7r krb5kdc[354](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474512, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM
I20260811 18:55:12.419195 507 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cddb4b6c9c5d4f8f8160335cc9dc22fd: Neither blocks nor log segments found. Creating new log.
I20260811 18:55:12.419996 507 log.cc:824] T 00000000000000000000000000000000 P cddb4b6c9c5d4f8f8160335cc9dc22fd: Log is configured to *not* fsync() on all Append() calls
I20260811 18:55:12.422293 507 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cddb4b6c9c5d4f8f8160335cc9dc22fd: No bootstrap required, opened a new log
I20260811 18:55:12.422992 512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:12.409756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58154 (local address 127.27.180.190:45397)
0811 18:55:12.410178 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:12.410196 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:12.410238 (+ 42us) server_negotiation.cc:407] Connection header received
0811 18:55:12.411061 (+ 823us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:12.411082 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:12.411500 (+ 418us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:12.411854 (+ 354us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:12.412480 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:12.413963 (+ 1483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:12.414761 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:12.415451 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:12.415603 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:12.418497 (+ 2894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:12.418502 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:12.418514 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:12.418542 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:12.420158 (+ 1616us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:12.420647 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:12.420653 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:12.420659 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:12.420801 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:12.421148 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:12.421150 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:12.421151 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:12.421525 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:12.421675 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:12.422001 (+ 326us) server_negotiation.cc:299] Negotiation successful
0811 18:55:12.422250 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":116,"threads_started":1}
I20260811 18:55:12.426290 507 raft_consensus.cc:348] T 00000000000000000000000000000000 P cddb4b6c9c5d4f8f8160335cc9dc22fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cddb4b6c9c5d4f8f8160335cc9dc22fd" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 45397 } }
I20260811 18:55:12.426523 507 raft_consensus.cc:374] T 00000000000000000000000000000000 P cddb4b6c9c5d4f8f8160335cc9dc22fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:55:12.426584 507 raft_consensus.cc:729] T 00000000000000000000000000000000 P cddb4b6c9c5d4f8f8160335cc9dc22fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cddb4b6c9c5d4f8f8160335cc9dc22fd, State: Initialized, Role: FOLLOWER
I20260811 18:55:12.426999 507 consensus_queue.cc:260] T 00000000000000000000000000000000 P cddb4b6c9c5d4f8f8160335cc9dc22fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cddb4b6c9c5d4f8f8160335cc9dc22fd" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 45397 } }
I20260811 18:55:12.427134 507 raft_consensus.cc:388] T 00000000000000000000000000000000 P cddb4b6c9c5d4f8f8160335cc9dc22fd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:55:12.427202 507 raft_consensus.cc:482] T 00000000000000000000000000000000 P cddb4b6c9c5d4f8f8160335cc9dc22fd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:55:12.427290 507 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cddb4b6c9c5d4f8f8160335cc9dc22fd [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:55:12.428293 507 raft_consensus.cc:504] T 00000000000000000000000000000000 P cddb4b6c9c5d4f8f8160335cc9dc22fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cddb4b6c9c5d4f8f8160335cc9dc22fd" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 45397 } }
I20260811 18:55:12.428743 507 leader_election.cc:302] T 00000000000000000000000000000000 P cddb4b6c9c5d4f8f8160335cc9dc22fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cddb4b6c9c5d4f8f8160335cc9dc22fd; no voters:
I20260811 18:55:12.429155 507 leader_election.cc:288] T 00000000000000000000000000000000 P cddb4b6c9c5d4f8f8160335cc9dc22fd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:55:12.429406 515 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cddb4b6c9c5d4f8f8160335cc9dc22fd [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:55:12.429612 515 raft_consensus.cc:686] T 00000000000000000000000000000000 P cddb4b6c9c5d4f8f8160335cc9dc22fd [term 1 LEADER]: Becoming Leader. State: Replica: cddb4b6c9c5d4f8f8160335cc9dc22fd, State: Running, Role: LEADER
I20260811 18:55:12.429951 515 consensus_queue.cc:237] T 00000000000000000000000000000000 P cddb4b6c9c5d4f8f8160335cc9dc22fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cddb4b6c9c5d4f8f8160335cc9dc22fd" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 45397 } }
I20260811 18:55:12.430616 507 sys_catalog.cc:564] T 00000000000000000000000000000000 P cddb4b6c9c5d4f8f8160335cc9dc22fd [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:55:12.431676 524 sys_catalog.cc:455] T 00000000000000000000000000000000 P cddb4b6c9c5d4f8f8160335cc9dc22fd [sys.catalog]: SysCatalogTable state changed. Reason: New leader cddb4b6c9c5d4f8f8160335cc9dc22fd. Latest consensus state: current_term: 1 leader_uuid: "cddb4b6c9c5d4f8f8160335cc9dc22fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cddb4b6c9c5d4f8f8160335cc9dc22fd" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 45397 } } }
I20260811 18:55:12.431800 524 sys_catalog.cc:458] T 00000000000000000000000000000000 P cddb4b6c9c5d4f8f8160335cc9dc22fd [sys.catalog]: This master's current role is: LEADER
I20260811 18:55:12.432241 540 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:55:12.432200 522 sys_catalog.cc:455] T 00000000000000000000000000000000 P cddb4b6c9c5d4f8f8160335cc9dc22fd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cddb4b6c9c5d4f8f8160335cc9dc22fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cddb4b6c9c5d4f8f8160335cc9dc22fd" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 45397 } } }
I20260811 18:55:12.432377 522 sys_catalog.cc:458] T 00000000000000000000000000000000 P cddb4b6c9c5d4f8f8160335cc9dc22fd [sys.catalog]: This master's current role is: LEADER
I20260811 18:55:12.435346 540 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:55:12.440712 540 catalog_manager.cc:1347] Generated new cluster ID: 25813f78254c496b8cf4a8d3edb875bf
I20260811 18:55:12.440786 540 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:55:12.456152 540 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:55:12.457115 540 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:55:12.470470 540 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cddb4b6c9c5d4f8f8160335cc9dc22fd: Generated new TSK 0
I20260811 18:55:12.471156 540 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.180.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.180.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.180.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.129.keytab.
Entry for principal kudu/127.27.180.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.129.keytab.
I20260811 18:55:12.504753 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474496157106-28370-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474496157106-28370-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474496157106-28370-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474496157106-28370-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.180.129:0
--local_ip_for_outbound_sockets=127.27.180.129
--webserver_interface=127.27.180.129
--webserver_port=0
--tserver_master_addrs=127.27.180.190:45397
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.129.keytab
--principal=kudu/127.27.180.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:39419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:55:12.616776 555 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:55:12.617023 555 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:55:12.617066 555 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:55:12.621035 555 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:55:12.621104 555 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:55:12.621176 555 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.180.129
I20260811 18:55:12.626549 555 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:39419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474496157106-28370-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474496157106-28370-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.129.keytab
--principal=kudu/127.27.180.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.180.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474496157106-28370-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.180.129
--webserver_port=0
--tserver_master_addrs=127.27.180.190:45397
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.555
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.180.129
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474496157106-28370-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:55:12.627528 555 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:55:12.628500 555 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:55:12.639549 567 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:12.639593 568 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:12.639637 571 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:55:12.639978 555 server_base.cc:1034] running on GCE node
I20260811 18:55:12.640370 555 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:55:12.641170 555 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:55:12.642362 555 hybrid_clock.cc:648] HybridClock initialized: now 1786474512642340 us; error 27 us; skew 500 ppm
Aug 11 18:55:12 dist-test-slave-2j7r krb5kdc[354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474512, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:55:12.648064 555 init.cc:377] Logged in from keytab as kudu/127.27.180.129@KRBTEST.COM (short username kudu)
I20260811 18:55:12.650748 555 webserver.cc:460] Webserver started at http://127.27.180.129:43603/ using document root <none> and password file <none>
I20260811 18:55:12.651456 555 fs_manager.cc:359] Metadata directory not provided
I20260811 18:55:12.651588 555 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:55:12.651870 555 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:55:12.654659 555 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474496157106-28370-0/minicluster-data/ts-0/data/instance:
uuid: "90740b5aa1fe42ad9265f959bb1a8195"
format_stamp: "Formatted at 2026-08-11 18:55:12 on dist-test-slave-2j7r"
server_key: "117428bc78321c9d06c5c1d84278df4a"
server_key_iv: "a183dc23e5bafd62de62236f1a951427"
server_key_version: "encryptionkey@0"
I20260811 18:55:12.655429 555 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474496157106-28370-0/minicluster-data/ts-0/wal/instance:
uuid: "90740b5aa1fe42ad9265f959bb1a8195"
format_stamp: "Formatted at 2026-08-11 18:55:12 on dist-test-slave-2j7r"
server_key: "117428bc78321c9d06c5c1d84278df4a"
server_key_iv: "a183dc23e5bafd62de62236f1a951427"
server_key_version: "encryptionkey@0"
I20260811 18:55:12.663460 555 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.007s sys 0.001s
I20260811 18:55:12.666646 583 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:12.668315 555 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 18:55:12.668498 555 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474496157106-28370-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474496157106-28370-0/minicluster-data/ts-0/wal
uuid: "90740b5aa1fe42ad9265f959bb1a8195"
format_stamp: "Formatted at 2026-08-11 18:55:12 on dist-test-slave-2j7r"
server_key: "117428bc78321c9d06c5c1d84278df4a"
server_key_iv: "a183dc23e5bafd62de62236f1a951427"
server_key_version: "encryptionkey@0"
I20260811 18:55:12.668669 555 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474496157106-28370-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474496157106-28370-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474496157106-28370-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:55:12.686822 555 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:55:12.689448 555 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:55:12.689594 555 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:55:12.690114 555 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:55:12.690903 555 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:55:12.690949 555 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:12.690989 555 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:55:12.691016 555 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:12.699144 555 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.129:34273
I20260811 18:55:12.699229 804 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.129:34273 every 8 connection(s)
I20260811 18:55:12.700004 555 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474496157106-28370-0/minicluster-data/ts-0/data/info.pb
I20260811 18:55:12.703049 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 555
I20260811 18:55:12.703176 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474496157106-28370-0/minicluster-data/ts-0/wal/instance
I20260811 18:55:12.703521 28370 external_mini_cluster.cc:1428] Setting key 3b5e0296521836b72cefebf26852f560
Aug 11 18:55:12 dist-test-slave-2j7r krb5kdc[354](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474512, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.129@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.27.180.130@KRBTEST.COM; defaulting to no policy
I20260811 18:55:12.713986 512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:12.701723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.129:50685 (local address 127.27.180.190:45397)
0811 18:55:12.701842 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:12.701846 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:12.702677 (+ 831us) server_negotiation.cc:407] Connection header received
0811 18:55:12.704284 (+ 1607us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:12.704287 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:12.704347 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:12.704431 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:12.705797 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:12.706366 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:12.707677 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:12.708022 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:12.708133 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:12.710648 (+ 2515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:12.710651 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:12.710653 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:12.710676 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:12.712000 (+ 1324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:12.712492 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:12.712495 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:12.712496 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:12.712574 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:12.712899 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:12.712902 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:12.712904 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:12.713079 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:12.713201 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:12.713687 (+ 486us) server_negotiation.cc:299] Negotiation successful
0811 18:55:12.713817 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 18:55:12.714746 808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:12.702050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:45397 (local address 127.27.180.129:50685)
0811 18:55:12.702515 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:12.702557 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:12.704078 (+ 1521us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:12.704594 (+ 516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:12.704601 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:12.704988 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:12.705644 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:12.705664 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:12.706496 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:12.706499 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:12.707554 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:12.707564 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:12.708154 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:12.708156 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:12.708317 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:12.708875 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:12.708902 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:12.710489 (+ 1587us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:55:12.712111 (+ 1622us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:12.712117 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:12.712131 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:12.712394 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:12.712657 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:12.712661 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:12.712663 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:12.712808 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:12.713222 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:55:12.713228 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:55:12.713458 (+ 230us) client_negotiation.cc:769] Sending connection context
0811 18:55:12.713625 (+ 167us) client_negotiation.cc:240] Negotiation successful
0811 18:55:12.713944 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":120,"threads_started":1}
I20260811 18:55:12.715902 805 heartbeater.cc:347] Connected to a master server at 127.27.180.190:45397
I20260811 18:55:12.716166 805 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.180.130@KRBTEST.COM" created.
I20260811 18:55:12.716713 805 heartbeater.cc:511] Master 127.27.180.190:45397 requested a full tablet report, sending...
I20260811 18:55:12.718380 448 ts_manager.cc:194] Registered new tserver with Master: 90740b5aa1fe42ad9265f959bb1a8195 (127.27.180.129:34273)
I20260811 18:55:12.719475 448 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.180.129@KRBTEST.COM'} at 127.27.180.129:50685
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.180.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.130.keytab.
Entry for principal kudu/127.27.180.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.130.keytab.
I20260811 18:55:12.731580 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474496157106-28370-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474496157106-28370-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474496157106-28370-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474496157106-28370-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.180.130:0
--local_ip_for_outbound_sockets=127.27.180.130
--webserver_interface=127.27.180.130
--webserver_port=0
--tserver_master_addrs=127.27.180.190:45397
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.130.keytab
--principal=kudu/127.27.180.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:39419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:55:12.839062 813 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:55:12.839282 813 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:55:12.839321 813 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:55:12.843226 813 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:55:12.843277 813 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:55:12.843333 813 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.180.130
I20260811 18:55:12.848335 813 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:39419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474496157106-28370-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474496157106-28370-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.130.keytab
--principal=kudu/127.27.180.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.180.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474496157106-28370-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.180.130
--webserver_port=0
--tserver_master_addrs=127.27.180.190:45397
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.813
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.180.130
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474496157106-28370-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:55:12.849273 813 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:55:12.850196 813 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:55:12.859287 827 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:55:12.859380 813 server_base.cc:1034] running on GCE node
W20260811 18:55:12.859320 826 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:12.859328 830 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:55:12.859932 813 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:55:12.860499 813 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:55:12.861649 813 hybrid_clock.cc:648] HybridClock initialized: now 1786474512861629 us; error 33 us; skew 500 ppm
Aug 11 18:55:12 dist-test-slave-2j7r krb5kdc[354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474512, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:55:12.864360 813 init.cc:377] Logged in from keytab as kudu/127.27.180.130@KRBTEST.COM (short username kudu)
I20260811 18:55:12.865321 813 webserver.cc:460] Webserver started at http://127.27.180.130:34715/ using document root <none> and password file <none>
I20260811 18:55:12.865980 813 fs_manager.cc:359] Metadata directory not provided
I20260811 18:55:12.866067 813 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:55:12.866386 813 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:55:12.868592 813 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474496157106-28370-0/minicluster-data/ts-1/data/instance:
uuid: "f15018b27aa3461d87daf82ce2a8bfc5"
format_stamp: "Formatted at 2026-08-11 18:55:12 on dist-test-slave-2j7r"
server_key: "80c1a3c01803d7415c06a1df4a6e74f6"
server_key_iv: "c8739139be9e59d784508e8097b950f3"
server_key_version: "encryptionkey@0"
I20260811 18:55:12.869165 813 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474496157106-28370-0/minicluster-data/ts-1/wal/instance:
uuid: "f15018b27aa3461d87daf82ce2a8bfc5"
format_stamp: "Formatted at 2026-08-11 18:55:12 on dist-test-slave-2j7r"
server_key: "80c1a3c01803d7415c06a1df4a6e74f6"
server_key_iv: "c8739139be9e59d784508e8097b950f3"
server_key_version: "encryptionkey@0"
I20260811 18:55:12.872603 813 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260811 18:55:12.874922 843 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:12.875991 813 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 18:55:12.876093 813 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474496157106-28370-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474496157106-28370-0/minicluster-data/ts-1/wal
uuid: "f15018b27aa3461d87daf82ce2a8bfc5"
format_stamp: "Formatted at 2026-08-11 18:55:12 on dist-test-slave-2j7r"
server_key: "80c1a3c01803d7415c06a1df4a6e74f6"
server_key_iv: "c8739139be9e59d784508e8097b950f3"
server_key_version: "encryptionkey@0"
I20260811 18:55:12.876178 813 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474496157106-28370-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474496157106-28370-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474496157106-28370-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:55:12.885046 813 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:55:12.887825 813 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:55:12.887987 813 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:55:12.888489 813 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:55:12.889277 813 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:55:12.889323 813 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:12.889374 813 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:55:12.889402 813 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:12.897555 813 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.130:34851
I20260811 18:55:12.897583 1056 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.130:34851 every 8 connection(s)
I20260811 18:55:12.898332 813 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474496157106-28370-0/minicluster-data/ts-1/data/info.pb
Aug 11 18:55:12 dist-test-slave-2j7r krb5kdc[354](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474512, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.130@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM
I20260811 18:55:12.907474 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 813
I20260811 18:55:12.907567 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474496157106-28370-0/minicluster-data/ts-1/wal/instance
I20260811 18:55:12.907826 28370 external_mini_cluster.cc:1428] Setting key aaeb89ea3229fd6b762c8bf560445edc
I20260811 18:55:12.910969 512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:12.899951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.130:44515 (local address 127.27.180.190:45397)
0811 18:55:12.900072 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:12.900076 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:12.900837 (+ 761us) server_negotiation.cc:407] Connection header received
0811 18:55:12.902124 (+ 1287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:12.902128 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:12.902185 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:12.902256 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:12.903348 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:12.903869 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:12.905069 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:12.905330 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:12.905417 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:12.907667 (+ 2250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:12.907671 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:12.907673 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:12.907698 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:12.908990 (+ 1292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:12.909568 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:12.909571 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:12.909572 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:12.909653 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:12.910002 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:12.910004 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:12.910005 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:12.910156 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:12.910231 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:12.910761 (+ 530us) server_negotiation.cc:299] Negotiation successful
0811 18:55:12.910854 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 18:55:12.911760 1060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:12.900281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:45397 (local address 127.27.180.130:44515)
0811 18:55:12.900696 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:12.900735 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:12.901926 (+ 1191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:12.902452 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:12.902459 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:12.902778 (+ 319us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:12.903203 (+ 425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:12.903215 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:12.903986 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:12.903988 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:12.904949 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:12.904960 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:12.905479 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:12.905483 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:12.905631 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:12.906085 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:12.906100 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:12.907507 (+ 1407us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:55:12.909121 (+ 1614us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:12.909127 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:12.909139 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:12.909455 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:12.909735 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:12.909739 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:12.909741 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:12.909924 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:12.910239 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:55:12.910245 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:55:12.910514 (+ 269us) client_negotiation.cc:769] Sending connection context
0811 18:55:12.910697 (+ 183us) client_negotiation.cc:240] Negotiation successful
0811 18:55:12.910998 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":105,"threads_started":1}
I20260811 18:55:12.912791 1057 heartbeater.cc:347] Connected to a master server at 127.27.180.190:45397
I20260811 18:55:12.913013 1057 heartbeater.cc:464] Registering TS with master...
I20260811 18:55:12.913527 1057 heartbeater.cc:511] Master 127.27.180.190:45397 requested a full tablet report, sending...
I20260811 18:55:12.914620 448 ts_manager.cc:194] Registered new tserver with Master: f15018b27aa3461d87daf82ce2a8bfc5 (127.27.180.130:34851)
I20260811 18:55:12.915149 448 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.180.130@KRBTEST.COM'} at 127.27.180.130:44515
WARNING: no policy specified for kudu/127.27.180.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.180.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.180.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.131.keytab.
Entry for principal kudu/127.27.180.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.131.keytab.
I20260811 18:55:12.934235 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474496157106-28370-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474496157106-28370-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474496157106-28370-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474496157106-28370-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.180.131:0
--local_ip_for_outbound_sockets=127.27.180.131
--webserver_interface=127.27.180.131
--webserver_port=0
--tserver_master_addrs=127.27.180.190:45397
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.131.keytab
--principal=kudu/127.27.180.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:39419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:55:13.043028 1064 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:55:13.043264 1064 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:55:13.043319 1064 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:55:13.047290 1064 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:55:13.047343 1064 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:55:13.047420 1064 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.180.131
I20260811 18:55:13.052474 1064 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:39419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474496157106-28370-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474496157106-28370-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.131.keytab
--principal=kudu/127.27.180.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.180.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474496157106-28370-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.180.131
--webserver_port=0
--tserver_master_addrs=127.27.180.190:45397
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1064
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.180.131
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474496157106-28370-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:55:13.053453 1064 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:55:13.054401 1064 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:55:13.063723 1069 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:13.063793 1073 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:13.063863 1070 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:55:13.064085 1064 server_base.cc:1034] running on GCE node
I20260811 18:55:13.064453 1064 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:55:13.065053 1064 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:55:13.066287 1064 hybrid_clock.cc:648] HybridClock initialized: now 1786474513066270 us; error 26 us; skew 500 ppm
Aug 11 18:55:13 dist-test-slave-2j7r krb5kdc[354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474513, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:55:13.068913 1064 init.cc:377] Logged in from keytab as kudu/127.27.180.131@KRBTEST.COM (short username kudu)
I20260811 18:55:13.069746 1064 webserver.cc:460] Webserver started at http://127.27.180.131:43401/ using document root <none> and password file <none>
I20260811 18:55:13.070227 1064 fs_manager.cc:359] Metadata directory not provided
I20260811 18:55:13.070284 1064 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:55:13.070446 1064 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:55:13.072340 1064 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474496157106-28370-0/minicluster-data/ts-2/data/instance:
uuid: "48543ca7d3914f69be6b318c07edce7f"
format_stamp: "Formatted at 2026-08-11 18:55:13 on dist-test-slave-2j7r"
server_key: "1b09c15ee78528bffe1727eba4a4b280"
server_key_iv: "14ddb328f386579365677c435e64c08d"
server_key_version: "encryptionkey@0"
I20260811 18:55:13.072847 1064 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474496157106-28370-0/minicluster-data/ts-2/wal/instance:
uuid: "48543ca7d3914f69be6b318c07edce7f"
format_stamp: "Formatted at 2026-08-11 18:55:13 on dist-test-slave-2j7r"
server_key: "1b09c15ee78528bffe1727eba4a4b280"
server_key_iv: "14ddb328f386579365677c435e64c08d"
server_key_version: "encryptionkey@0"
I20260811 18:55:13.076351 1064 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260811 18:55:13.078612 1085 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:13.079800 1064 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 18:55:13.079905 1064 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474496157106-28370-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474496157106-28370-0/minicluster-data/ts-2/wal
uuid: "48543ca7d3914f69be6b318c07edce7f"
format_stamp: "Formatted at 2026-08-11 18:55:13 on dist-test-slave-2j7r"
server_key: "1b09c15ee78528bffe1727eba4a4b280"
server_key_iv: "14ddb328f386579365677c435e64c08d"
server_key_version: "encryptionkey@0"
I20260811 18:55:13.079995 1064 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474496157106-28370-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474496157106-28370-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474496157106-28370-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:55:13.096866 1064 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:55:13.099387 1064 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:55:13.099545 1064 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:55:13.100085 1064 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:55:13.100850 1064 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:55:13.100895 1064 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:13.100936 1064 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:55:13.100967 1064 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:13.109397 1064 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.131:40391
I20260811 18:55:13.109452 1318 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.131:40391 every 8 connection(s)
I20260811 18:55:13.110219 1064 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474496157106-28370-0/minicluster-data/ts-2/data/info.pb
Aug 11 18:55:13 dist-test-slave-2j7r krb5kdc[354](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474513, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.131@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM
I20260811 18:55:13.119647 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 1064
I20260811 18:55:13.119768 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786474496157106-28370-0/minicluster-data/ts-2/wal/instance
I20260811 18:55:13.120097 28370 external_mini_cluster.cc:1428] Setting key 3123eb74cdaf0295d43d0dc18e8e98aa
I20260811 18:55:13.123159 512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:13.111911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.131:34561 (local address 127.27.180.190:45397)
0811 18:55:13.112035 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:13.112038 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:13.112822 (+ 784us) server_negotiation.cc:407] Connection header received
0811 18:55:13.114134 (+ 1312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:13.114137 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:13.114199 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:13.114271 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:13.115391 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:13.115919 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:13.117018 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:13.117284 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:13.117377 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:13.119663 (+ 2286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:13.119668 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:13.119671 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:13.119702 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:13.121306 (+ 1604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:13.121803 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:13.121807 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:13.121808 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:13.121917 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:13.122261 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:13.122263 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:13.122265 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:13.122448 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:13.122527 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:13.122946 (+ 419us) server_negotiation.cc:299] Negotiation successful
0811 18:55:13.123041 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 18:55:13.123994 1322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:13.112253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:45397 (local address 127.27.180.131:34561)
0811 18:55:13.112674 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:13.112714 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:13.113923 (+ 1209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:13.114462 (+ 539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:13.114469 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:13.114803 (+ 334us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:13.115248 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:13.115261 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:13.116037 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:13.116040 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:13.116900 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:13.116909 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:13.117437 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:13.117442 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:13.117599 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:13.118099 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:13.118114 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:13.119497 (+ 1383us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:55:13.121402 (+ 1905us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:13.121408 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:13.121422 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:13.121698 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:13.122008 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:13.122011 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:13.122013 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:13.122176 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:13.122524 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:55:13.122529 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:55:13.122757 (+ 228us) client_negotiation.cc:769] Sending connection context
0811 18:55:13.122905 (+ 148us) client_negotiation.cc:240] Negotiation successful
0811 18:55:13.123141 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":104,"threads_started":1}
I20260811 18:55:13.125026 1319 heartbeater.cc:347] Connected to a master server at 127.27.180.190:45397
I20260811 18:55:13.125274 1319 heartbeater.cc:464] Registering TS with master...
I20260811 18:55:13.125741 1319 heartbeater.cc:511] Master 127.27.180.190:45397 requested a full tablet report, sending...
I20260811 18:55:13.126736 448 ts_manager.cc:194] Registered new tserver with Master: 48543ca7d3914f69be6b318c07edce7f (127.27.180.131:40391)
I20260811 18:55:13.127315 448 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.180.131@KRBTEST.COM'} at 127.27.180.131:34561
I20260811 18:55:13.133715 28370 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 18:55:13.147764 512 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:55:13.146404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58162 (local address 127.27.180.190:45397)
0811 18:55:13.146554 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:13.146559 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:13.146686 (+ 127us) server_negotiation.cc:407] Connection header received
0811 18:55:13.146841 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:13.146843 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:13.146885 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:13.146947 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:13.147656 (+ 709us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58162: BlockingRecv error: recv got EOF from 127.0.0.1:58162 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 18:55:13.148893 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 555
I20260811 18:55:13.166499 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 813
I20260811 18:55:13.177482 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 1064
I20260811 18:55:13.204510 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 383
2026-08-11T18:55:13Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2279 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474496157106-28370-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:55:13 dist-test-slave-2j7r krb5kdc[1342](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:55:13 dist-test-slave-2j7r krb5kdc[1342](info): set up 2 sockets
Aug 11 18:55:13 dist-test-slave-2j7r krb5kdc[1342](info): commencing operation
krb5kdc: starting...
W20260811 18:55:14.398820 28370 mini_kdc.cc:121] Time spent starting KDC: real 1.168s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474496157106-28370-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474496157106-28370-0/krb5kdc/test-user.keytab.
Aug 11 18:55:14 dist-test-slave-2j7r krb5kdc[1342](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474514, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:55:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:55:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.180.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.180.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.180.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab.
Entry for principal kudu/127.27.180.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab.
I20260811 18:55:14.521834 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474496157106-28370-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474496157106-28370-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474496157106-28370-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.190:41367
--webserver_interface=127.27.180.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab
--principal=kudu/127.27.180.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:40427
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.180.190:41367
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:55:14.630318 1364 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:55:14.630565 1364 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:55:14.630604 1364 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:55:14.634501 1364 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:55:14.634549 1364 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:55:14.634567 1364 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:55:14.634583 1364 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:55:14.634596 1364 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:55:14.639564 1364 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:40427
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474496157106-28370-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474496157106-28370-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.180.190:41367
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab
--principal=kudu/127.27.180.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.190:41367
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.180.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1364
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474496157106-28370-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:55:14.640466 1364 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:55:14.641490 1364 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:55:14.649721 1386 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:14.649729 1384 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:55:14.649928 1364 server_base.cc:1034] running on GCE node
W20260811 18:55:14.649778 1391 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:55:14.650563 1364 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:55:14.651626 1364 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:55:14.652799 1364 hybrid_clock.cc:648] HybridClock initialized: now 1786474514652780 us; error 32 us; skew 500 ppm
Aug 11 18:55:14 dist-test-slave-2j7r krb5kdc[1342](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474514, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:55:14.659551 1364 init.cc:377] Logged in from keytab as kudu/127.27.180.190@KRBTEST.COM (short username kudu)
I20260811 18:55:14.660415 1364 webserver.cc:460] Webserver started at http://127.27.180.190:42035/ using document root <none> and password file <none>
I20260811 18:55:14.660934 1364 fs_manager.cc:359] Metadata directory not provided
I20260811 18:55:14.661000 1364 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:55:14.661230 1364 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:55:14.663218 1364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474496157106-28370-0/minicluster-data/master-0/data/instance:
uuid: "3167879518c74d5e944a042243db6127"
format_stamp: "Formatted at 2026-08-11 18:55:14 on dist-test-slave-2j7r"
server_key: "567340bcb5acd4e6953187d822b94c3a"
server_key_iv: "de5c81db19717a3e840150ea5c93e573"
server_key_version: "encryptionkey@0"
I20260811 18:55:14.663847 1364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474496157106-28370-0/minicluster-data/master-0/wal/instance:
uuid: "3167879518c74d5e944a042243db6127"
format_stamp: "Formatted at 2026-08-11 18:55:14 on dist-test-slave-2j7r"
server_key: "567340bcb5acd4e6953187d822b94c3a"
server_key_iv: "de5c81db19717a3e840150ea5c93e573"
server_key_version: "encryptionkey@0"
I20260811 18:55:14.667508 1364 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260811 18:55:14.670054 1405 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:14.671231 1364 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260811 18:55:14.671345 1364 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474496157106-28370-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474496157106-28370-0/minicluster-data/master-0/wal
uuid: "3167879518c74d5e944a042243db6127"
format_stamp: "Formatted at 2026-08-11 18:55:14 on dist-test-slave-2j7r"
server_key: "567340bcb5acd4e6953187d822b94c3a"
server_key_iv: "de5c81db19717a3e840150ea5c93e573"
server_key_version: "encryptionkey@0"
I20260811 18:55:14.671456 1364 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474496157106-28370-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474496157106-28370-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474496157106-28370-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:55:14.690976 1364 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:55:14.694047 1364 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:55:14.694228 1364 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:55:14.701151 1364 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.190:41367
I20260811 18:55:14.701426 1499 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.190:41367 every 8 connection(s)
I20260811 18:55:14.702118 1364 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
I20260811 18:55:14.704905 1500 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:55:14.708033 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 1364
I20260811 18:55:14.708137 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786474496157106-28370-0/minicluster-data/master-0/wal/instance
I20260811 18:55:14.708387 28370 external_mini_cluster.cc:1428] Setting key 7c596a969f86feccbf1badf208936610
I20260811 18:55:14.710965 1500 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3167879518c74d5e944a042243db6127: Bootstrap starting.
I20260811 18:55:14.713160 1500 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3167879518c74d5e944a042243db6127: Neither blocks nor log segments found. Creating new log.
I20260811 18:55:14.714021 1500 log.cc:824] T 00000000000000000000000000000000 P 3167879518c74d5e944a042243db6127: Log is configured to *not* fsync() on all Append() calls
I20260811 18:55:14.716305 1500 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3167879518c74d5e944a042243db6127: No bootstrap required, opened a new log
Aug 11 18:55:14 dist-test-slave-2j7r krb5kdc[1342](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474514, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM
I20260811 18:55:14.719571 1500 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3167879518c74d5e944a042243db6127 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3167879518c74d5e944a042243db6127" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 41367 } }
I20260811 18:55:14.719794 1500 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3167879518c74d5e944a042243db6127 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:55:14.719856 1500 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3167879518c74d5e944a042243db6127 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3167879518c74d5e944a042243db6127, State: Initialized, Role: FOLLOWER
I20260811 18:55:14.720438 1500 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3167879518c74d5e944a042243db6127 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3167879518c74d5e944a042243db6127" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 41367 } }
I20260811 18:55:14.720606 1500 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3167879518c74d5e944a042243db6127 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:55:14.720675 1500 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3167879518c74d5e944a042243db6127 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:55:14.720770 1500 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3167879518c74d5e944a042243db6127 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:55:14.721666 1500 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3167879518c74d5e944a042243db6127 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3167879518c74d5e944a042243db6127" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 41367 } }
I20260811 18:55:14.722023 1500 leader_election.cc:302] T 00000000000000000000000000000000 P 3167879518c74d5e944a042243db6127 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3167879518c74d5e944a042243db6127; no voters:
I20260811 18:55:14.722374 1500 leader_election.cc:288] T 00000000000000000000000000000000 P 3167879518c74d5e944a042243db6127 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:55:14.722470 1509 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3167879518c74d5e944a042243db6127 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:55:14.722671 1509 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3167879518c74d5e944a042243db6127 [term 1 LEADER]: Becoming Leader. State: Replica: 3167879518c74d5e944a042243db6127, State: Running, Role: LEADER
I20260811 18:55:14.723011 1507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:14.709738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55442 (local address 127.27.180.190:41367)
0811 18:55:14.710190 (+ 452us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:14.710204 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:14.710242 (+ 38us) server_negotiation.cc:407] Connection header received
0811 18:55:14.710848 (+ 606us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:14.710863 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:14.711203 (+ 340us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:14.711511 (+ 308us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:14.712334 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:14.713369 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:14.714241 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:14.714859 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:14.714988 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:14.717684 (+ 2696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:14.717689 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:14.717701 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:14.717728 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:14.719885 (+ 2157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:14.720441 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:14.720446 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:14.720450 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:14.720557 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:14.720925 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:14.720929 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:14.720931 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:14.721357 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:14.721532 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:14.721829 (+ 297us) server_negotiation.cc:299] Negotiation successful
0811 18:55:14.722112 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":139,"threads_started":1}
I20260811 18:55:14.723045 1509 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3167879518c74d5e944a042243db6127 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3167879518c74d5e944a042243db6127" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 41367 } }
I20260811 18:55:14.723690 1500 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3167879518c74d5e944a042243db6127 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:55:14.725415 1513 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3167879518c74d5e944a042243db6127 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3167879518c74d5e944a042243db6127. Latest consensus state: current_term: 1 leader_uuid: "3167879518c74d5e944a042243db6127" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3167879518c74d5e944a042243db6127" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 41367 } } }
I20260811 18:55:14.725945 28370 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:55:14.725669 1510 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3167879518c74d5e944a042243db6127 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3167879518c74d5e944a042243db6127" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3167879518c74d5e944a042243db6127" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 41367 } } }
I20260811 18:55:14.726204 1510 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3167879518c74d5e944a042243db6127 [sys.catalog]: This master's current role is: LEADER
I20260811 18:55:14.725772 1513 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3167879518c74d5e944a042243db6127 [sys.catalog]: This master's current role is: LEADER
W20260811 18:55:14.728209 1537 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3167879518c74d5e944a042243db6127: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:55:14.728273 1537 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:55:14.728354 1539 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:55:14.729260 1539 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:55:14.734954 1539 catalog_manager.cc:1347] Generated new cluster ID: 307fcecdb13646179130ba9cd4677458
I20260811 18:55:14.735047 1539 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:55:14.745791 1539 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:55:14.747014 1539 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:55:14.747025 1507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:14.740805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55448 (local address 127.27.180.190:41367)
0811 18:55:14.740915 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:14.740921 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:14.741208 (+ 287us) server_negotiation.cc:407] Connection header received
0811 18:55:14.741342 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:14.741344 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:14.741394 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:14.741470 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:14.742053 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:14.742847 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:14.743995 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:14.744328 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:14.744431 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:14.746035 (+ 1604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:14.746038 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:14.746041 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:55:14.746588 (+ 547us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:14.746622 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:55:14.746642 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:14.746735 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:14.746839 (+ 104us) server_negotiation.cc:299] Negotiation successful
0811 18:55:14.746893 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 18:55:14.752362 1539 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3167879518c74d5e944a042243db6127: Generated new TSK 0
I20260811 18:55:14.753062 1539 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:55:14.759276 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 1364
2026-08-11T18:55:14Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1555 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474496157106-28370-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:55:14 dist-test-slave-2j7r krb5kdc[1571](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:55:14 dist-test-slave-2j7r krb5kdc[1571](info): set up 2 sockets
Aug 11 18:55:14 dist-test-slave-2j7r krb5kdc[1571](info): commencing operation
krb5kdc: starting...
W20260811 18:55:15.955192 28370 mini_kdc.cc:121] Time spent starting KDC: real 1.169s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474496157106-28370-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474496157106-28370-0/krb5kdc/test-user.keytab.
Aug 11 18:55:16 dist-test-slave-2j7r krb5kdc[1571](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474516, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:55:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:55:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.180.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.180.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.180.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab.
Entry for principal kudu/127.27.180.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab.
I20260811 18:55:16.074146 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474496157106-28370-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474496157106-28370-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474496157106-28370-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.190:45913
--webserver_interface=127.27.180.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab
--principal=kudu/127.27.180.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:35345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.180.190:45913
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:55:16.182034 1592 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:55:16.182260 1592 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:55:16.182299 1592 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:55:16.186225 1592 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:55:16.186272 1592 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:55:16.186290 1592 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:55:16.186304 1592 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:55:16.186318 1592 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:55:16.191443 1592 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:35345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474496157106-28370-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474496157106-28370-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.180.190:45913
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab
--principal=kudu/127.27.180.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.190:45913
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.180.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1592
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474496157106-28370-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:55:16.192370 1592 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:55:16.193359 1592 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:55:16.201453 1611 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:16.201519 1606 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:16.201453 1605 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:55:16.201702 1592 server_base.cc:1034] running on GCE node
I20260811 18:55:16.202334 1592 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:55:16.203362 1592 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:55:16.204516 1592 hybrid_clock.cc:648] HybridClock initialized: now 1786474516204495 us; error 36 us; skew 500 ppm
Aug 11 18:55:16 dist-test-slave-2j7r krb5kdc[1571](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474516, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:55:16.207173 1592 init.cc:377] Logged in from keytab as kudu/127.27.180.190@KRBTEST.COM (short username kudu)
I20260811 18:55:16.208043 1592 webserver.cc:460] Webserver started at http://127.27.180.190:43767/ using document root <none> and password file <none>
I20260811 18:55:16.208566 1592 fs_manager.cc:359] Metadata directory not provided
I20260811 18:55:16.208631 1592 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:55:16.208818 1592 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:55:16.210862 1592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474496157106-28370-0/minicluster-data/master-0/data/instance:
uuid: "ab15ea58e37343f3bdc24ca096c9bdb5"
format_stamp: "Formatted at 2026-08-11 18:55:16 on dist-test-slave-2j7r"
server_key: "602e884a36458af7da6b5643d7b6ba51"
server_key_iv: "4b23188a592d4e7c998901696e8e9121"
server_key_version: "encryptionkey@0"
I20260811 18:55:16.211395 1592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474496157106-28370-0/minicluster-data/master-0/wal/instance:
uuid: "ab15ea58e37343f3bdc24ca096c9bdb5"
format_stamp: "Formatted at 2026-08-11 18:55:16 on dist-test-slave-2j7r"
server_key: "602e884a36458af7da6b5643d7b6ba51"
server_key_iv: "4b23188a592d4e7c998901696e8e9121"
server_key_version: "encryptionkey@0"
I20260811 18:55:16.215042 1592 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 18:55:16.217370 1628 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:16.218493 1592 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 18:55:16.218604 1592 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474496157106-28370-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474496157106-28370-0/minicluster-data/master-0/wal
uuid: "ab15ea58e37343f3bdc24ca096c9bdb5"
format_stamp: "Formatted at 2026-08-11 18:55:16 on dist-test-slave-2j7r"
server_key: "602e884a36458af7da6b5643d7b6ba51"
server_key_iv: "4b23188a592d4e7c998901696e8e9121"
server_key_version: "encryptionkey@0"
I20260811 18:55:16.218690 1592 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474496157106-28370-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474496157106-28370-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474496157106-28370-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:55:16.233865 1592 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:55:16.236559 1592 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:55:16.236729 1592 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:55:16.243374 1592 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.190:45913
I20260811 18:55:16.243681 1727 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.190:45913 every 8 connection(s)
I20260811 18:55:16.244280 1592 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
I20260811 18:55:16.246997 1731 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:55:16.249620 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 1592
I20260811 18:55:16.249750 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474496157106-28370-0/minicluster-data/master-0/wal/instance
I20260811 18:55:16.250054 28370 external_mini_cluster.cc:1428] Setting key 4a04a2601c6fa0ddf0417c69fd9c907b
I20260811 18:55:16.253108 1731 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab15ea58e37343f3bdc24ca096c9bdb5: Bootstrap starting.
I20260811 18:55:16.255297 1731 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ab15ea58e37343f3bdc24ca096c9bdb5: Neither blocks nor log segments found. Creating new log.
I20260811 18:55:16.255996 1731 log.cc:824] T 00000000000000000000000000000000 P ab15ea58e37343f3bdc24ca096c9bdb5: Log is configured to *not* fsync() on all Append() calls
I20260811 18:55:16.257885 1731 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab15ea58e37343f3bdc24ca096c9bdb5: No bootstrap required, opened a new log
Aug 11 18:55:16 dist-test-slave-2j7r krb5kdc[1571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474516, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM
I20260811 18:55:16.261059 1731 raft_consensus.cc:348] T 00000000000000000000000000000000 P ab15ea58e37343f3bdc24ca096c9bdb5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab15ea58e37343f3bdc24ca096c9bdb5" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 45913 } }
I20260811 18:55:16.261307 1731 raft_consensus.cc:374] T 00000000000000000000000000000000 P ab15ea58e37343f3bdc24ca096c9bdb5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:55:16.261366 1731 raft_consensus.cc:729] T 00000000000000000000000000000000 P ab15ea58e37343f3bdc24ca096c9bdb5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab15ea58e37343f3bdc24ca096c9bdb5, State: Initialized, Role: FOLLOWER
I20260811 18:55:16.261798 1731 consensus_queue.cc:260] T 00000000000000000000000000000000 P ab15ea58e37343f3bdc24ca096c9bdb5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab15ea58e37343f3bdc24ca096c9bdb5" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 45913 } }
I20260811 18:55:16.261929 1731 raft_consensus.cc:388] T 00000000000000000000000000000000 P ab15ea58e37343f3bdc24ca096c9bdb5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:55:16.261991 1731 raft_consensus.cc:482] T 00000000000000000000000000000000 P ab15ea58e37343f3bdc24ca096c9bdb5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:55:16.262084 1731 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ab15ea58e37343f3bdc24ca096c9bdb5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:55:16.262887 1731 raft_consensus.cc:504] T 00000000000000000000000000000000 P ab15ea58e37343f3bdc24ca096c9bdb5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab15ea58e37343f3bdc24ca096c9bdb5" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 45913 } }
I20260811 18:55:16.263198 1731 leader_election.cc:302] T 00000000000000000000000000000000 P ab15ea58e37343f3bdc24ca096c9bdb5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ab15ea58e37343f3bdc24ca096c9bdb5; no voters:
I20260811 18:55:16.263466 1731 leader_election.cc:288] T 00000000000000000000000000000000 P ab15ea58e37343f3bdc24ca096c9bdb5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:55:16.263563 1745 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ab15ea58e37343f3bdc24ca096c9bdb5 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:55:16.263813 1745 raft_consensus.cc:686] T 00000000000000000000000000000000 P ab15ea58e37343f3bdc24ca096c9bdb5 [term 1 LEADER]: Becoming Leader. State: Replica: ab15ea58e37343f3bdc24ca096c9bdb5, State: Running, Role: LEADER
I20260811 18:55:16.264196 1745 consensus_queue.cc:237] T 00000000000000000000000000000000 P ab15ea58e37343f3bdc24ca096c9bdb5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab15ea58e37343f3bdc24ca096c9bdb5" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 45913 } }
I20260811 18:55:16.264267 1741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:16.251480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37976 (local address 127.27.180.190:45913)
0811 18:55:16.251918 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:16.251933 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:16.251968 (+ 35us) server_negotiation.cc:407] Connection header received
0811 18:55:16.252581 (+ 613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:16.252597 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:16.252914 (+ 317us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:16.253237 (+ 323us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:16.254036 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:16.255019 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:16.255902 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:16.256404 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:16.256552 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:16.259067 (+ 2515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:16.259072 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:16.259084 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:16.259111 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:16.261153 (+ 2042us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:16.261620 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:16.261626 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:16.261630 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:16.261747 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:16.262028 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:16.262032 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:16.262033 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:16.262444 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:16.262639 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:16.262914 (+ 275us) server_negotiation.cc:299] Negotiation successful
0811 18:55:16.263303 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":131,"threads_started":1}
I20260811 18:55:16.264678 1731 sys_catalog.cc:564] T 00000000000000000000000000000000 P ab15ea58e37343f3bdc24ca096c9bdb5 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:55:16.265413 1747 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab15ea58e37343f3bdc24ca096c9bdb5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ab15ea58e37343f3bdc24ca096c9bdb5. Latest consensus state: current_term: 1 leader_uuid: "ab15ea58e37343f3bdc24ca096c9bdb5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab15ea58e37343f3bdc24ca096c9bdb5" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 45913 } } }
I20260811 18:55:16.265604 1747 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab15ea58e37343f3bdc24ca096c9bdb5 [sys.catalog]: This master's current role is: LEADER
I20260811 18:55:16.265887 1748 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab15ea58e37343f3bdc24ca096c9bdb5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ab15ea58e37343f3bdc24ca096c9bdb5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab15ea58e37343f3bdc24ca096c9bdb5" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 45913 } } }
I20260811 18:55:16.266053 1748 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab15ea58e37343f3bdc24ca096c9bdb5 [sys.catalog]: This master's current role is: LEADER
I20260811 18:55:16.266513 1773 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:55:16.269586 1773 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:55:16.275056 1773 catalog_manager.cc:1347] Generated new cluster ID: a02fb0edcd884937a784f9695a1df76d
I20260811 18:55:16.275125 1773 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:55:16.292973 1773 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:55:16.294198 1773 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:55:16.305176 1773 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ab15ea58e37343f3bdc24ca096c9bdb5: Generated new TSK 0
I20260811 18:55:16.305862 1773 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.180.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.180.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.180.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.129.keytab.
Entry for principal kudu/127.27.180.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.129.keytab.
I20260811 18:55:16.341411 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474496157106-28370-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474496157106-28370-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474496157106-28370-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474496157106-28370-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.180.129:0
--local_ip_for_outbound_sockets=127.27.180.129
--webserver_interface=127.27.180.129
--webserver_port=0
--tserver_master_addrs=127.27.180.190:45913
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.129.keytab
--principal=kudu/127.27.180.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:35345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:55:16.481945 1792 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:55:16.482198 1792 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:55:16.482257 1792 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:55:16.486245 1792 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:55:16.486326 1792 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:55:16.486402 1792 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.180.129
I20260811 18:55:16.492826 1792 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:35345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474496157106-28370-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474496157106-28370-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.129.keytab
--principal=kudu/127.27.180.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.180.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474496157106-28370-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.180.129
--webserver_port=0
--tserver_master_addrs=127.27.180.190:45913
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1792
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.180.129
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474496157106-28370-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:55:16.493911 1792 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:55:16.494953 1792 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 18:55:16.504676 1792 server_base.cc:1034] running on GCE node
W20260811 18:55:16.504613 1800 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:16.504613 1799 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:16.504833 1804 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:55:16.505249 1792 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:55:16.505857 1792 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:55:16.507036 1792 hybrid_clock.cc:648] HybridClock initialized: now 1786474516507019 us; error 29 us; skew 500 ppm
Aug 11 18:55:16 dist-test-slave-2j7r krb5kdc[1571](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474516, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:55:16.509814 1792 init.cc:377] Logged in from keytab as kudu/127.27.180.129@KRBTEST.COM (short username kudu)
I20260811 18:55:16.510726 1792 webserver.cc:460] Webserver started at http://127.27.180.129:34115/ using document root <none> and password file <none>
I20260811 18:55:16.511248 1792 fs_manager.cc:359] Metadata directory not provided
I20260811 18:55:16.511308 1792 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:55:16.511472 1792 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:55:16.513525 1792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474496157106-28370-0/minicluster-data/ts-0/data/instance:
uuid: "cde08045c5e0493e8f463e2409b374fb"
format_stamp: "Formatted at 2026-08-11 18:55:16 on dist-test-slave-2j7r"
server_key: "2bdb10a9302aab88b6f4ad97621913be"
server_key_iv: "a7f9cdced5374f37c0bbdef94d05e96a"
server_key_version: "encryptionkey@0"
I20260811 18:55:16.514071 1792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474496157106-28370-0/minicluster-data/ts-0/wal/instance:
uuid: "cde08045c5e0493e8f463e2409b374fb"
format_stamp: "Formatted at 2026-08-11 18:55:16 on dist-test-slave-2j7r"
server_key: "2bdb10a9302aab88b6f4ad97621913be"
server_key_iv: "a7f9cdced5374f37c0bbdef94d05e96a"
server_key_version: "encryptionkey@0"
I20260811 18:55:16.517843 1792 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 18:55:16.520391 1817 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:16.521736 1792 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 18:55:16.521855 1792 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474496157106-28370-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474496157106-28370-0/minicluster-data/ts-0/wal
uuid: "cde08045c5e0493e8f463e2409b374fb"
format_stamp: "Formatted at 2026-08-11 18:55:16 on dist-test-slave-2j7r"
server_key: "2bdb10a9302aab88b6f4ad97621913be"
server_key_iv: "a7f9cdced5374f37c0bbdef94d05e96a"
server_key_version: "encryptionkey@0"
I20260811 18:55:16.521953 1792 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474496157106-28370-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474496157106-28370-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474496157106-28370-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:55:16.552120 1792 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:55:16.555518 1792 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:55:16.555704 1792 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:55:16.556310 1792 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:55:16.557195 1792 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:55:16.557250 1792 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:16.557294 1792 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:55:16.557318 1792 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:16.566483 1792 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.129:38483
I20260811 18:55:16.566511 2029 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.129:38483 every 8 connection(s)
I20260811 18:55:16.567386 1792 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474496157106-28370-0/minicluster-data/ts-0/data/info.pb
I20260811 18:55:16.568609 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 1792
I20260811 18:55:16.568727 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474496157106-28370-0/minicluster-data/ts-0/wal/instance
I20260811 18:55:16.569016 28370 external_mini_cluster.cc:1428] Setting key 01f13a831a0081a29cde87bd48333994
Aug 11 18:55:16 dist-test-slave-2j7r krb5kdc[1571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474516, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.129@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.27.180.130@KRBTEST.COM; defaulting to no policy
I20260811 18:55:16.581239 1741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:16.569197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.129:52487 (local address 127.27.180.190:45913)
0811 18:55:16.569353 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:16.569358 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:16.570205 (+ 847us) server_negotiation.cc:407] Connection header received
0811 18:55:16.571683 (+ 1478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:16.571686 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:16.571740 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:16.571813 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:16.572957 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:16.573546 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:16.575053 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:16.575309 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:16.575401 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:16.578088 (+ 2687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:16.578091 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:16.578093 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:16.578117 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:16.579410 (+ 1293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:16.579914 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:16.579916 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:16.579918 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:16.579992 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:16.580250 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:16.580252 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:16.580253 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:16.580484 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:16.580574 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:16.580987 (+ 413us) server_negotiation.cc:299] Negotiation successful
0811 18:55:16.581105 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 18:55:16.582054 2033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:16.569595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:45913 (local address 127.27.180.129:52487)
0811 18:55:16.570043 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:16.570095 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:16.571473 (+ 1378us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:16.571992 (+ 519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:16.571998 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:16.572334 (+ 336us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:16.572802 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:16.572816 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:16.573672 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:16.573676 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:16.574922 (+ 1246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:16.574934 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:16.575483 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:16.575488 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:16.575683 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:16.576233 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:16.576254 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:16.577925 (+ 1671us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:55:16.579535 (+ 1610us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:16.579542 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:16.579557 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:16.579830 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:16.580057 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:16.580059 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:16.580060 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:16.580184 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:16.580577 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:55:16.580581 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:55:16.580798 (+ 217us) client_negotiation.cc:769] Sending connection context
0811 18:55:16.580951 (+ 153us) client_negotiation.cc:240] Negotiation successful
0811 18:55:16.581232 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":112,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.180.130@KRBTEST.COM" created.
I20260811 18:55:16.583387 2030 heartbeater.cc:347] Connected to a master server at 127.27.180.190:45913
I20260811 18:55:16.583647 2030 heartbeater.cc:464] Registering TS with master...
I20260811 18:55:16.584156 2030 heartbeater.cc:511] Master 127.27.180.190:45913 requested a full tablet report, sending...
I20260811 18:55:16.585816 1655 ts_manager.cc:194] Registered new tserver with Master: cde08045c5e0493e8f463e2409b374fb (127.27.180.129:38483)
I20260811 18:55:16.587419 1655 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.180.129@KRBTEST.COM'} at 127.27.180.129:52487
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.180.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.130.keytab.
Entry for principal kudu/127.27.180.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.130.keytab.
I20260811 18:55:16.597919 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474496157106-28370-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474496157106-28370-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474496157106-28370-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474496157106-28370-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.180.130:0
--local_ip_for_outbound_sockets=127.27.180.130
--webserver_interface=127.27.180.130
--webserver_port=0
--tserver_master_addrs=127.27.180.190:45913
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.130.keytab
--principal=kudu/127.27.180.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:35345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:55:16.706403 2036 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:55:16.706631 2036 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:55:16.706672 2036 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:55:16.710589 2036 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:55:16.710638 2036 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:55:16.710693 2036 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.180.130
I20260811 18:55:16.715888 2036 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:35345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474496157106-28370-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474496157106-28370-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.130.keytab
--principal=kudu/127.27.180.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.180.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474496157106-28370-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.180.130
--webserver_port=0
--tserver_master_addrs=127.27.180.190:45913
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2036
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.180.130
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474496157106-28370-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:55:16.717124 2036 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:55:16.718076 2036 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:55:16.727524 2044 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:16.727524 2046 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:55:16.727689 2036 server_base.cc:1034] running on GCE node
W20260811 18:55:16.728479 2043 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:55:16.728902 2036 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:55:16.729624 2036 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:55:16.730813 2036 hybrid_clock.cc:648] HybridClock initialized: now 1786474516730798 us; error 23 us; skew 500 ppm
Aug 11 18:55:16 dist-test-slave-2j7r krb5kdc[1571](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474516, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:55:16.733671 2036 init.cc:377] Logged in from keytab as kudu/127.27.180.130@KRBTEST.COM (short username kudu)
I20260811 18:55:16.734489 2036 webserver.cc:460] Webserver started at http://127.27.180.130:41109/ using document root <none> and password file <none>
I20260811 18:55:16.734988 2036 fs_manager.cc:359] Metadata directory not provided
I20260811 18:55:16.735050 2036 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:55:16.735231 2036 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:55:16.737249 2036 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474496157106-28370-0/minicluster-data/ts-1/data/instance:
uuid: "ef5af607dbfc40578fb4461b51b78345"
format_stamp: "Formatted at 2026-08-11 18:55:16 on dist-test-slave-2j7r"
server_key: "6b0f1956234a408776946070432ea781"
server_key_iv: "69f32ac0273c13962ee8e8e9e135db75"
server_key_version: "encryptionkey@0"
I20260811 18:55:16.737783 2036 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474496157106-28370-0/minicluster-data/ts-1/wal/instance:
uuid: "ef5af607dbfc40578fb4461b51b78345"
format_stamp: "Formatted at 2026-08-11 18:55:16 on dist-test-slave-2j7r"
server_key: "6b0f1956234a408776946070432ea781"
server_key_iv: "69f32ac0273c13962ee8e8e9e135db75"
server_key_version: "encryptionkey@0"
I20260811 18:55:16.741649 2036 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 18:55:16.743981 2072 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:16.745231 2036 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 18:55:16.745349 2036 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474496157106-28370-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474496157106-28370-0/minicluster-data/ts-1/wal
uuid: "ef5af607dbfc40578fb4461b51b78345"
format_stamp: "Formatted at 2026-08-11 18:55:16 on dist-test-slave-2j7r"
server_key: "6b0f1956234a408776946070432ea781"
server_key_iv: "69f32ac0273c13962ee8e8e9e135db75"
server_key_version: "encryptionkey@0"
I20260811 18:55:16.745446 2036 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474496157106-28370-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474496157106-28370-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474496157106-28370-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:55:16.756647 2036 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:55:16.759397 2036 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:55:16.759538 2036 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:55:16.760016 2036 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:55:16.760751 2036 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:55:16.760795 2036 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:16.760831 2036 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:55:16.760852 2036 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:16.769188 2036 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.130:36493
I20260811 18:55:16.769236 2297 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.130:36493 every 8 connection(s)
I20260811 18:55:16.769970 2036 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474496157106-28370-0/minicluster-data/ts-1/data/info.pb
I20260811 18:55:16.773852 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 2036
I20260811 18:55:16.773970 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474496157106-28370-0/minicluster-data/ts-1/wal/instance
I20260811 18:55:16.774273 28370 external_mini_cluster.cc:1428] Setting key 4125337c09606aad5cbe4a5a69048dab
Aug 11 18:55:16 dist-test-slave-2j7r krb5kdc[1571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474516, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.130@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM
I20260811 18:55:16.783278 1741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:16.771537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.130:52161 (local address 127.27.180.190:45913)
0811 18:55:16.771660 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:16.771663 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:16.772413 (+ 750us) server_negotiation.cc:407] Connection header received
0811 18:55:16.773810 (+ 1397us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:16.773813 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:16.773861 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:16.773950 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:16.775212 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:16.775785 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:16.777247 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:16.777473 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:16.777552 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:16.779964 (+ 2412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:16.779967 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:16.779969 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:16.779993 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:16.781297 (+ 1304us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:16.781866 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:16.781868 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:16.781870 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:16.781934 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:16.782278 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:16.782280 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:16.782281 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:16.782440 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:16.782511 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:16.783061 (+ 550us) server_negotiation.cc:299] Negotiation successful
0811 18:55:16.783149 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 18:55:16.784349 2302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:16.771872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:45913 (local address 127.27.180.130:52161)
0811 18:55:16.772283 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:16.772321 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:16.773595 (+ 1274us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:16.774113 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:16.774122 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:16.774583 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:16.775088 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:16.775105 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:16.775903 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:16.775906 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:16.777133 (+ 1227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:16.777144 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:16.777621 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:16.777625 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:16.777813 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:16.778311 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:16.778327 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:16.779800 (+ 1473us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:55:16.781406 (+ 1606us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:16.781412 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:16.781426 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:16.781777 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:16.782014 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:16.782017 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:16.782019 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:16.782196 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:16.782532 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:55:16.782538 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:55:16.782826 (+ 288us) client_negotiation.cc:769] Sending connection context
0811 18:55:16.783042 (+ 216us) client_negotiation.cc:240] Negotiation successful
0811 18:55:16.783354 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":104,"threads_started":1}
WARNING: no policy specified for kudu/127.27.180.131@KRBTEST.COM; defaulting to no policy
I20260811 18:55:16.785383 2298 heartbeater.cc:347] Connected to a master server at 127.27.180.190:45913
I20260811 18:55:16.785631 2298 heartbeater.cc:464] Registering TS with master...
I20260811 18:55:16.786131 2298 heartbeater.cc:511] Master 127.27.180.190:45913 requested a full tablet report, sending...
I20260811 18:55:16.787096 1655 ts_manager.cc:194] Registered new tserver with Master: ef5af607dbfc40578fb4461b51b78345 (127.27.180.130:36493)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.180.131@KRBTEST.COM" created.
I20260811 18:55:16.787750 1655 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.180.130@KRBTEST.COM'} at 127.27.180.130:52161
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.180.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.131.keytab.
Entry for principal kudu/127.27.180.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.131.keytab.
I20260811 18:55:16.802106 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474496157106-28370-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474496157106-28370-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474496157106-28370-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474496157106-28370-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.180.131:0
--local_ip_for_outbound_sockets=127.27.180.131
--webserver_interface=127.27.180.131
--webserver_port=0
--tserver_master_addrs=127.27.180.190:45913
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.131.keytab
--principal=kudu/127.27.180.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:35345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:55:16.910991 2307 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:55:16.911218 2307 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:55:16.911284 2307 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:55:16.915196 2307 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:55:16.915256 2307 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:55:16.915324 2307 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.180.131
I20260811 18:55:16.920337 2307 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:35345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474496157106-28370-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474496157106-28370-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.131.keytab
--principal=kudu/127.27.180.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.180.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474496157106-28370-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.180.131
--webserver_port=0
--tserver_master_addrs=127.27.180.190:45913
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2307
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.180.131
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474496157106-28370-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:55:16.921262 2307 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:55:16.922192 2307 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:55:16.931212 2316 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:16.931182 2314 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:55:16.931386 2307 server_base.cc:1034] running on GCE node
W20260811 18:55:16.931182 2312 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:55:16.931960 2307 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:55:16.932528 2307 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:55:16.933681 2307 hybrid_clock.cc:648] HybridClock initialized: now 1786474516933666 us; error 29 us; skew 500 ppm
Aug 11 18:55:16 dist-test-slave-2j7r krb5kdc[1571](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474516, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:55:16.936331 2307 init.cc:377] Logged in from keytab as kudu/127.27.180.131@KRBTEST.COM (short username kudu)
I20260811 18:55:16.937106 2307 webserver.cc:460] Webserver started at http://127.27.180.131:44799/ using document root <none> and password file <none>
I20260811 18:55:16.937645 2307 fs_manager.cc:359] Metadata directory not provided
I20260811 18:55:16.937706 2307 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:55:16.937875 2307 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:55:16.939792 2307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474496157106-28370-0/minicluster-data/ts-2/data/instance:
uuid: "cc54f5ea15d04991858d6796bdf8f7a5"
format_stamp: "Formatted at 2026-08-11 18:55:16 on dist-test-slave-2j7r"
server_key: "3d8c13e33f964f59959f6f872218417e"
server_key_iv: "426d8ba5f1c9f6ed248e19e17289a2ce"
server_key_version: "encryptionkey@0"
I20260811 18:55:16.940294 2307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474496157106-28370-0/minicluster-data/ts-2/wal/instance:
uuid: "cc54f5ea15d04991858d6796bdf8f7a5"
format_stamp: "Formatted at 2026-08-11 18:55:16 on dist-test-slave-2j7r"
server_key: "3d8c13e33f964f59959f6f872218417e"
server_key_iv: "426d8ba5f1c9f6ed248e19e17289a2ce"
server_key_version: "encryptionkey@0"
I20260811 18:55:16.943773 2307 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260811 18:55:16.946069 2330 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:16.947405 2307 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 18:55:16.947516 2307 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474496157106-28370-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474496157106-28370-0/minicluster-data/ts-2/wal
uuid: "cc54f5ea15d04991858d6796bdf8f7a5"
format_stamp: "Formatted at 2026-08-11 18:55:16 on dist-test-slave-2j7r"
server_key: "3d8c13e33f964f59959f6f872218417e"
server_key_iv: "426d8ba5f1c9f6ed248e19e17289a2ce"
server_key_version: "encryptionkey@0"
I20260811 18:55:16.947623 2307 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474496157106-28370-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474496157106-28370-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474496157106-28370-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:55:16.972095 2307 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:55:16.974967 2307 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:55:16.975154 2307 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:55:16.975728 2307 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:55:16.976532 2307 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:55:16.976579 2307 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:16.976632 2307 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:55:16.976660 2307 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:16.984922 2307 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.131:40417
I20260811 18:55:16.984957 2532 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.131:40417 every 8 connection(s)
I20260811 18:55:16.985790 2307 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474496157106-28370-0/minicluster-data/ts-2/data/info.pb
I20260811 18:55:16.988106 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 2307
I20260811 18:55:16.988240 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786474496157106-28370-0/minicluster-data/ts-2/wal/instance
I20260811 18:55:16.988579 28370 external_mini_cluster.cc:1428] Setting key 17a639c915bc6573bfb545ad08326b54
Aug 11 18:55:16 dist-test-slave-2j7r krb5kdc[1571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474516, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.131@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM
I20260811 18:55:16.999538 1741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:16.987594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.131:43455 (local address 127.27.180.190:45913)
0811 18:55:16.987741 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:16.987745 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:16.988648 (+ 903us) server_negotiation.cc:407] Connection header received
0811 18:55:16.990224 (+ 1576us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:16.990228 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:16.990283 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:16.990358 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:16.991524 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:16.992055 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:16.993211 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:16.993445 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:16.993537 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:16.996022 (+ 2485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:16.996025 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:16.996027 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:16.996055 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:16.997411 (+ 1356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:16.997927 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:16.997931 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:16.997933 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:16.998021 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:16.998360 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:16.998363 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:16.998365 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:16.998570 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:16.998659 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:16.999213 (+ 554us) server_negotiation.cc:299] Negotiation successful
0811 18:55:16.999354 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 18:55:17.000401 2538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:16.987968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:45913 (local address 127.27.180.131:43455)
0811 18:55:16.988517 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:16.988559 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:16.990025 (+ 1466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:16.990541 (+ 516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:16.990548 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:16.990911 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:16.991382 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:16.991396 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:16.992174 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:16.992177 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:16.993069 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:16.993080 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:16.993593 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:16.993598 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:16.993770 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:16.994326 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:16.994344 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:16.995840 (+ 1496us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:55:16.997525 (+ 1685us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:16.997532 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:16.997550 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:16.997826 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:16.998111 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:16.998114 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:16.998116 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:16.998268 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:16.998695 (+ 427us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:55:16.998702 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:55:16.998971 (+ 269us) client_negotiation.cc:769] Sending connection context
0811 18:55:16.999148 (+ 177us) client_negotiation.cc:240] Negotiation successful
0811 18:55:16.999515 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":193,"threads_started":1}
I20260811 18:55:17.001716 2535 heartbeater.cc:347] Connected to a master server at 127.27.180.190:45913
I20260811 18:55:17.001959 2535 heartbeater.cc:464] Registering TS with master...
I20260811 18:55:17.002508 2535 heartbeater.cc:511] Master 127.27.180.190:45913 requested a full tablet report, sending...
I20260811 18:55:17.003495 1655 ts_manager.cc:194] Registered new tserver with Master: cc54f5ea15d04991858d6796bdf8f7a5 (127.27.180.131:40417)
I20260811 18:55:17.004055 1655 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.180.131@KRBTEST.COM'} at 127.27.180.131:43455
I20260811 18:55:17.014163 28370 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 18:55:17 dist-test-slave-2j7r krb5kdc[1571](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474517, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 11 18:55:17 dist-test-slave-2j7r krb5kdc[1571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474517, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM
I20260811 18:55:17.039932 1741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:17.031664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52370 (local address 127.27.180.190:45913)
0811 18:55:17.031792 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:17.031797 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:17.031972 (+ 175us) server_negotiation.cc:407] Connection header received
0811 18:55:17.032105 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:17.032109 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:17.032155 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:17.032223 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:17.032911 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:17.033480 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:17.034307 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:17.034533 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:17.034623 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:17.036829 (+ 2206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:17.036832 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:17.036834 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:17.036859 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:17.038052 (+ 1193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:17.038555 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:17.038559 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:17.038560 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:17.038742 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:17.039070 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:17.039073 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:17.039075 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:17.039337 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:17.039458 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:17.039688 (+ 230us) server_negotiation.cc:299] Negotiation successful
0811 18:55:17.039791 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
W20260811 18:55:17.040361 1655 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:52370
I20260811 18:55:17.041194 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 1792
I20260811 18:55:17.065426 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 2036
I20260811 18:55:17.087280 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 2307
I20260811 18:55:17.104169 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 1592
2026-08-11T18:55:17Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2348 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474496157106-28370-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:55:17 dist-test-slave-2j7r krb5kdc[2553](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:55:17 dist-test-slave-2j7r krb5kdc[2553](info): set up 2 sockets
Aug 11 18:55:17 dist-test-slave-2j7r krb5kdc[2553](info): commencing operation
krb5kdc: starting...
W20260811 18:55:18.307902 28370 mini_kdc.cc:121] Time spent starting KDC: real 1.173s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474496157106-28370-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474496157106-28370-0/krb5kdc/test-user.keytab.
Aug 11 18:55:18 dist-test-slave-2j7r krb5kdc[2553](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474518, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:55:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:55:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.180.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.180.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.180.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab.
Entry for principal kudu/127.27.180.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab.
I20260811 18:55:18.426219 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474496157106-28370-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474496157106-28370-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474496157106-28370-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.190:35851
--webserver_interface=127.27.180.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab
--principal=kudu/127.27.180.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:41971
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.180.190:35851
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:55:18.535107 2580 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:55:18.535354 2580 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:55:18.535409 2580 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:55:18.539325 2580 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:55:18.539382 2580 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:55:18.539408 2580 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:55:18.539430 2580 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:55:18.539458 2580 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:55:18.544550 2580 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:41971
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474496157106-28370-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474496157106-28370-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.180.190:35851
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab
--principal=kudu/127.27.180.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.190:35851
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.180.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2580
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474496157106-28370-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:55:18.545619 2580 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:55:18.546622 2580 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:55:18.554639 2593 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:18.554631 2597 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:18.554654 2594 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:55:18.554841 2580 server_base.cc:1034] running on GCE node
I20260811 18:55:18.555477 2580 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:55:18.556509 2580 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:55:18.557674 2580 hybrid_clock.cc:648] HybridClock initialized: now 1786474518557655 us; error 31 us; skew 500 ppm
Aug 11 18:55:18 dist-test-slave-2j7r krb5kdc[2553](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474518, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:55:18.560313 2580 init.cc:377] Logged in from keytab as kudu/127.27.180.190@KRBTEST.COM (short username kudu)
I20260811 18:55:18.561173 2580 webserver.cc:460] Webserver started at http://127.27.180.190:46393/ using document root <none> and password file <none>
I20260811 18:55:18.561683 2580 fs_manager.cc:359] Metadata directory not provided
I20260811 18:55:18.561748 2580 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:55:18.561936 2580 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:55:18.563848 2580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474496157106-28370-0/minicluster-data/master-0/data/instance:
uuid: "084f5c3a20b54bb3811566006e95b4e4"
format_stamp: "Formatted at 2026-08-11 18:55:18 on dist-test-slave-2j7r"
server_key: "4b5db4586a113ef89e0f8222abc96ef2"
server_key_iv: "b26aa6712ab2f8e12a2b5f69668f1375"
server_key_version: "encryptionkey@0"
I20260811 18:55:18.564384 2580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474496157106-28370-0/minicluster-data/master-0/wal/instance:
uuid: "084f5c3a20b54bb3811566006e95b4e4"
format_stamp: "Formatted at 2026-08-11 18:55:18 on dist-test-slave-2j7r"
server_key: "4b5db4586a113ef89e0f8222abc96ef2"
server_key_iv: "b26aa6712ab2f8e12a2b5f69668f1375"
server_key_version: "encryptionkey@0"
I20260811 18:55:18.567827 2580 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260811 18:55:18.570016 2612 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:18.571141 2580 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 18:55:18.571290 2580 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474496157106-28370-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474496157106-28370-0/minicluster-data/master-0/wal
uuid: "084f5c3a20b54bb3811566006e95b4e4"
format_stamp: "Formatted at 2026-08-11 18:55:18 on dist-test-slave-2j7r"
server_key: "4b5db4586a113ef89e0f8222abc96ef2"
server_key_iv: "b26aa6712ab2f8e12a2b5f69668f1375"
server_key_version: "encryptionkey@0"
I20260811 18:55:18.571395 2580 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474496157106-28370-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474496157106-28370-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474496157106-28370-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:55:18.586866 2580 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:55:18.589722 2580 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:55:18.589881 2580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:55:18.596309 2580 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.190:35851
I20260811 18:55:18.596326 2714 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.190:35851 every 8 connection(s)
I20260811 18:55:18.597182 2580 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
I20260811 18:55:18.599912 2715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:55:18.601398 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 2580
I20260811 18:55:18.601521 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474496157106-28370-0/minicluster-data/master-0/wal/instance
I20260811 18:55:18.601817 28370 external_mini_cluster.cc:1428] Setting key 61779e72403b14d2b425a80881e344d8
I20260811 18:55:18.605866 2715 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 084f5c3a20b54bb3811566006e95b4e4: Bootstrap starting.
I20260811 18:55:18.608263 2715 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 084f5c3a20b54bb3811566006e95b4e4: Neither blocks nor log segments found. Creating new log.
I20260811 18:55:18.609109 2715 log.cc:824] T 00000000000000000000000000000000 P 084f5c3a20b54bb3811566006e95b4e4: Log is configured to *not* fsync() on all Append() calls
Aug 11 18:55:18 dist-test-slave-2j7r krb5kdc[2553](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474518, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM
I20260811 18:55:18.611228 2715 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 084f5c3a20b54bb3811566006e95b4e4: No bootstrap required, opened a new log
I20260811 18:55:18.614919 2715 raft_consensus.cc:348] T 00000000000000000000000000000000 P 084f5c3a20b54bb3811566006e95b4e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "084f5c3a20b54bb3811566006e95b4e4" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 35851 } }
I20260811 18:55:18.615123 2715 raft_consensus.cc:374] T 00000000000000000000000000000000 P 084f5c3a20b54bb3811566006e95b4e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:55:18.615187 2715 raft_consensus.cc:729] T 00000000000000000000000000000000 P 084f5c3a20b54bb3811566006e95b4e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 084f5c3a20b54bb3811566006e95b4e4, State: Initialized, Role: FOLLOWER
I20260811 18:55:18.615319 2725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:18.603107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45958 (local address 127.27.180.190:35851)
0811 18:55:18.603485 (+ 378us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:18.603502 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:18.603536 (+ 34us) server_negotiation.cc:407] Connection header received
0811 18:55:18.604142 (+ 606us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:18.604157 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:18.604461 (+ 304us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:18.604711 (+ 250us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:18.605586 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:18.606573 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:18.607687 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:18.608130 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:18.608269 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:18.611004 (+ 2735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:18.611011 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:18.611026 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:18.611059 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:18.612612 (+ 1553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:18.613138 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:18.613145 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:18.613149 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:18.613267 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:18.613531 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:18.613533 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:18.613534 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:18.613884 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:18.614034 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:18.614306 (+ 272us) server_negotiation.cc:299] Negotiation successful
0811 18:55:18.614577 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":251,"thread_start_us":109,"threads_started":1}
I20260811 18:55:18.615630 2715 consensus_queue.cc:260] T 00000000000000000000000000000000 P 084f5c3a20b54bb3811566006e95b4e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "084f5c3a20b54bb3811566006e95b4e4" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 35851 } }
I20260811 18:55:18.615769 2715 raft_consensus.cc:388] T 00000000000000000000000000000000 P 084f5c3a20b54bb3811566006e95b4e4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:55:18.615833 2715 raft_consensus.cc:482] T 00000000000000000000000000000000 P 084f5c3a20b54bb3811566006e95b4e4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:55:18.615926 2715 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 084f5c3a20b54bb3811566006e95b4e4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:55:18.616896 2715 raft_consensus.cc:504] T 00000000000000000000000000000000 P 084f5c3a20b54bb3811566006e95b4e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "084f5c3a20b54bb3811566006e95b4e4" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 35851 } }
I20260811 18:55:18.617281 2715 leader_election.cc:302] T 00000000000000000000000000000000 P 084f5c3a20b54bb3811566006e95b4e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 084f5c3a20b54bb3811566006e95b4e4; no voters:
I20260811 18:55:18.617579 2715 leader_election.cc:288] T 00000000000000000000000000000000 P 084f5c3a20b54bb3811566006e95b4e4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:55:18.617669 2732 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 084f5c3a20b54bb3811566006e95b4e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:55:18.618037 2732 raft_consensus.cc:686] T 00000000000000000000000000000000 P 084f5c3a20b54bb3811566006e95b4e4 [term 1 LEADER]: Becoming Leader. State: Replica: 084f5c3a20b54bb3811566006e95b4e4, State: Running, Role: LEADER
I20260811 18:55:18.618356 2732 consensus_queue.cc:237] T 00000000000000000000000000000000 P 084f5c3a20b54bb3811566006e95b4e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "084f5c3a20b54bb3811566006e95b4e4" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 35851 } }
I20260811 18:55:18.618988 2715 sys_catalog.cc:564] T 00000000000000000000000000000000 P 084f5c3a20b54bb3811566006e95b4e4 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:55:18.620072 2734 sys_catalog.cc:455] T 00000000000000000000000000000000 P 084f5c3a20b54bb3811566006e95b4e4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "084f5c3a20b54bb3811566006e95b4e4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "084f5c3a20b54bb3811566006e95b4e4" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 35851 } } }
I20260811 18:55:18.620083 2733 sys_catalog.cc:455] T 00000000000000000000000000000000 P 084f5c3a20b54bb3811566006e95b4e4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 084f5c3a20b54bb3811566006e95b4e4. Latest consensus state: current_term: 1 leader_uuid: "084f5c3a20b54bb3811566006e95b4e4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "084f5c3a20b54bb3811566006e95b4e4" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 35851 } } }
I20260811 18:55:18.620290 2733 sys_catalog.cc:458] T 00000000000000000000000000000000 P 084f5c3a20b54bb3811566006e95b4e4 [sys.catalog]: This master's current role is: LEADER
I20260811 18:55:18.620283 2734 sys_catalog.cc:458] T 00000000000000000000000000000000 P 084f5c3a20b54bb3811566006e95b4e4 [sys.catalog]: This master's current role is: LEADER
I20260811 18:55:18.621050 2747 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:55:18.623941 2747 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:55:18.629153 2747 catalog_manager.cc:1347] Generated new cluster ID: 373a9949429d4e518c7345169b6d113d
I20260811 18:55:18.629215 2747 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:55:18.653936 2747 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:55:18.654836 2747 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:55:18.665699 2747 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 084f5c3a20b54bb3811566006e95b4e4: Generated new TSK 0
I20260811 18:55:18.666357 2747 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.180.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.180.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.180.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.129.keytab.
Entry for principal kudu/127.27.180.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.129.keytab.
I20260811 18:55:18.701787 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474496157106-28370-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474496157106-28370-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474496157106-28370-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474496157106-28370-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.180.129:0
--local_ip_for_outbound_sockets=127.27.180.129
--webserver_interface=127.27.180.129
--webserver_port=0
--tserver_master_addrs=127.27.180.190:35851
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.129.keytab
--principal=kudu/127.27.180.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:41971
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:55:18.810822 2779 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:55:18.811064 2779 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:55:18.811120 2779 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:55:18.815080 2779 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:55:18.815131 2779 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:55:18.815207 2779 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.180.129
I20260811 18:55:18.820360 2779 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:41971
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474496157106-28370-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474496157106-28370-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.129.keytab
--principal=kudu/127.27.180.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.180.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474496157106-28370-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.180.129
--webserver_port=0
--tserver_master_addrs=127.27.180.190:35851
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2779
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.180.129
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474496157106-28370-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:55:18.821410 2779 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:55:18.822366 2779 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:55:18.831482 2792 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:18.831488 2789 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:18.831534 2785 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:55:18.832228 2779 server_base.cc:1034] running on GCE node
I20260811 18:55:18.832577 2779 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:55:18.833184 2779 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:55:18.834355 2779 hybrid_clock.cc:648] HybridClock initialized: now 1786474518834331 us; error 39 us; skew 500 ppm
Aug 11 18:55:18 dist-test-slave-2j7r krb5kdc[2553](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474518, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:55:18.836903 2779 init.cc:377] Logged in from keytab as kudu/127.27.180.129@KRBTEST.COM (short username kudu)
I20260811 18:55:18.837734 2779 webserver.cc:460] Webserver started at http://127.27.180.129:40459/ using document root <none> and password file <none>
I20260811 18:55:18.838239 2779 fs_manager.cc:359] Metadata directory not provided
I20260811 18:55:18.838311 2779 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:55:18.838512 2779 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:55:18.840425 2779 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474496157106-28370-0/minicluster-data/ts-0/data/instance:
uuid: "f889543988d5472b973d315961a180d7"
format_stamp: "Formatted at 2026-08-11 18:55:18 on dist-test-slave-2j7r"
server_key: "183da75ed51063cd19e0ca2c0749aec2"
server_key_iv: "dc78e737e9043b2b23bbdf2721d5d737"
server_key_version: "encryptionkey@0"
I20260811 18:55:18.840953 2779 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474496157106-28370-0/minicluster-data/ts-0/wal/instance:
uuid: "f889543988d5472b973d315961a180d7"
format_stamp: "Formatted at 2026-08-11 18:55:18 on dist-test-slave-2j7r"
server_key: "183da75ed51063cd19e0ca2c0749aec2"
server_key_iv: "dc78e737e9043b2b23bbdf2721d5d737"
server_key_version: "encryptionkey@0"
I20260811 18:55:18.844383 2779 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.002s
I20260811 18:55:18.846675 2804 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:18.847790 2779 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 18:55:18.847892 2779 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474496157106-28370-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474496157106-28370-0/minicluster-data/ts-0/wal
uuid: "f889543988d5472b973d315961a180d7"
format_stamp: "Formatted at 2026-08-11 18:55:18 on dist-test-slave-2j7r"
server_key: "183da75ed51063cd19e0ca2c0749aec2"
server_key_iv: "dc78e737e9043b2b23bbdf2721d5d737"
server_key_version: "encryptionkey@0"
I20260811 18:55:18.847976 2779 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474496157106-28370-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474496157106-28370-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474496157106-28370-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:55:18.860258 2779 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:55:18.863330 2779 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:55:18.863508 2779 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:55:18.864114 2779 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:55:18.865018 2779 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:55:18.865067 2779 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:18.865140 2779 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:55:18.865176 2779 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:18.873202 2779 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.129:36991
I20260811 18:55:18.873257 3028 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.129:36991 every 8 connection(s)
I20260811 18:55:18.874030 2779 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474496157106-28370-0/minicluster-data/ts-0/data/info.pb
I20260811 18:55:18.878278 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 2779
I20260811 18:55:18.878394 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474496157106-28370-0/minicluster-data/ts-0/wal/instance
I20260811 18:55:18.878657 28370 external_mini_cluster.cc:1428] Setting key 32178d74ff3a49e733cae0062d6384e8
Aug 11 18:55:18 dist-test-slave-2j7r krb5kdc[2553](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474518, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.129@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM
I20260811 18:55:18.887003 2725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:18.875711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.129:44083 (local address 127.27.180.190:35851)
0811 18:55:18.875863 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:18.875869 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:18.876669 (+ 800us) server_negotiation.cc:407] Connection header received
0811 18:55:18.878103 (+ 1434us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:18.878107 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:18.878166 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:18.878240 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:18.879472 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:18.880082 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:18.881181 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:18.881408 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:18.881497 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:18.883671 (+ 2174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:18.883674 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:18.883677 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:18.883708 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:18.885053 (+ 1345us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:18.885567 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:18.885570 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:18.885571 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:18.885648 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:18.885964 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:18.885967 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:18.885969 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:18.886143 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:18.886228 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:18.886716 (+ 488us) server_negotiation.cc:299] Negotiation successful
0811 18:55:18.886844 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 18:55:18.887717 3033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:18.876058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:35851 (local address 127.27.180.129:44083)
0811 18:55:18.876515 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:18.876557 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:18.877898 (+ 1341us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:18.878435 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:18.878443 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:18.878811 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:18.879326 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:18.879342 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:18.880208 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:18.880211 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:18.881051 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:18.881060 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:18.881549 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:18.881554 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:18.881708 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:18.882196 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:18.882213 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:18.883522 (+ 1309us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:55:18.885176 (+ 1654us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:18.885182 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:18.885235 (+ 53us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:18.885472 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:18.885730 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:18.885733 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:18.885735 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:18.885865 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:18.886270 (+ 405us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:55:18.886276 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:55:18.886488 (+ 212us) client_negotiation.cc:769] Sending connection context
0811 18:55:18.886642 (+ 154us) client_negotiation.cc:240] Negotiation successful
0811 18:55:18.886946 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":113,"threads_started":1}
WARNING: no policy specified for kudu/127.27.180.130@KRBTEST.COM; defaulting to no policy
I20260811 18:55:18.888844 3031 heartbeater.cc:347] Connected to a master server at 127.27.180.190:35851
I20260811 18:55:18.889065 3031 heartbeater.cc:464] Registering TS with master...
I20260811 18:55:18.889585 3031 heartbeater.cc:511] Master 127.27.180.190:35851 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.180.130@KRBTEST.COM" created.
I20260811 18:55:18.891148 2638 ts_manager.cc:194] Registered new tserver with Master: f889543988d5472b973d315961a180d7 (127.27.180.129:36991)
I20260811 18:55:18.892247 2638 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.180.129@KRBTEST.COM'} at 127.27.180.129:44083
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.180.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.130.keytab.
Entry for principal kudu/127.27.180.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.130.keytab.
I20260811 18:55:18.905737 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474496157106-28370-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474496157106-28370-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474496157106-28370-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474496157106-28370-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.180.130:0
--local_ip_for_outbound_sockets=127.27.180.130
--webserver_interface=127.27.180.130
--webserver_port=0
--tserver_master_addrs=127.27.180.190:35851
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.130.keytab
--principal=kudu/127.27.180.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:41971
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:55:19.013024 3039 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:55:19.013263 3039 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:55:19.013317 3039 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:55:19.017163 3039 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:55:19.017204 3039 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:55:19.017256 3039 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.180.130
I20260811 18:55:19.022358 3039 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:41971
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474496157106-28370-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474496157106-28370-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.130.keytab
--principal=kudu/127.27.180.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.180.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474496157106-28370-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.180.130
--webserver_port=0
--tserver_master_addrs=127.27.180.190:35851
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3039
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.180.130
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474496157106-28370-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:55:19.023247 3039 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:55:19.024163 3039 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:55:19.033310 3059 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:19.033334 3054 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:19.033334 3056 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:55:19.033674 3039 server_base.cc:1034] running on GCE node
I20260811 18:55:19.034052 3039 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:55:19.034613 3039 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:55:19.035773 3039 hybrid_clock.cc:648] HybridClock initialized: now 1786474519035755 us; error 24 us; skew 500 ppm
Aug 11 18:55:19 dist-test-slave-2j7r krb5kdc[2553](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474519, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:55:19.038391 3039 init.cc:377] Logged in from keytab as kudu/127.27.180.130@KRBTEST.COM (short username kudu)
I20260811 18:55:19.039208 3039 webserver.cc:460] Webserver started at http://127.27.180.130:43681/ using document root <none> and password file <none>
I20260811 18:55:19.039719 3039 fs_manager.cc:359] Metadata directory not provided
I20260811 18:55:19.039793 3039 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:55:19.039991 3039 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:55:19.042058 3039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474496157106-28370-0/minicluster-data/ts-1/data/instance:
uuid: "4d99ebced7114e3d992946c70212eda9"
format_stamp: "Formatted at 2026-08-11 18:55:19 on dist-test-slave-2j7r"
server_key: "0bc0009bdc26c77ad6ced7b114d22524"
server_key_iv: "28b61ad5d45fe5b3876eeb0addd083be"
server_key_version: "encryptionkey@0"
I20260811 18:55:19.042582 3039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474496157106-28370-0/minicluster-data/ts-1/wal/instance:
uuid: "4d99ebced7114e3d992946c70212eda9"
format_stamp: "Formatted at 2026-08-11 18:55:19 on dist-test-slave-2j7r"
server_key: "0bc0009bdc26c77ad6ced7b114d22524"
server_key_iv: "28b61ad5d45fe5b3876eeb0addd083be"
server_key_version: "encryptionkey@0"
I20260811 18:55:19.046101 3039 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 18:55:19.048317 3069 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:19.049505 3039 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 18:55:19.049607 3039 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474496157106-28370-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474496157106-28370-0/minicluster-data/ts-1/wal
uuid: "4d99ebced7114e3d992946c70212eda9"
format_stamp: "Formatted at 2026-08-11 18:55:19 on dist-test-slave-2j7r"
server_key: "0bc0009bdc26c77ad6ced7b114d22524"
server_key_iv: "28b61ad5d45fe5b3876eeb0addd083be"
server_key_version: "encryptionkey@0"
I20260811 18:55:19.049703 3039 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474496157106-28370-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474496157106-28370-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474496157106-28370-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:55:19.061988 3039 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:55:19.064929 3039 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:55:19.065124 3039 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:55:19.065666 3039 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:55:19.066480 3039 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:55:19.066527 3039 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:19.066577 3039 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:55:19.066606 3039 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:19.074883 3039 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.130:41437
I20260811 18:55:19.074926 3321 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.130:41437 every 8 connection(s)
I20260811 18:55:19.075724 3039 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474496157106-28370-0/minicluster-data/ts-1/data/info.pb
I20260811 18:55:19.081362 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 3039
I20260811 18:55:19.081487 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474496157106-28370-0/minicluster-data/ts-1/wal/instance
I20260811 18:55:19.081784 28370 external_mini_cluster.cc:1428] Setting key 21ea2ab1f60ced50fce4fd9b3ef80f0e
Aug 11 18:55:19 dist-test-slave-2j7r krb5kdc[2553](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474519, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.130@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM
I20260811 18:55:19.089221 2725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:19.077486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.130:57595 (local address 127.27.180.190:35851)
0811 18:55:19.077632 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:19.077637 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:19.078477 (+ 840us) server_negotiation.cc:407] Connection header received
0811 18:55:19.079766 (+ 1289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:19.079768 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:19.079816 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:19.079890 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:19.080932 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:19.081661 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:19.082961 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:19.083217 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:19.083323 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:19.085828 (+ 2505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:19.085831 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:19.085833 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:19.085857 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:19.087282 (+ 1425us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:19.087742 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:19.087746 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:19.087747 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:19.087852 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:19.088157 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:19.088160 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:19.088161 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:19.088395 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:19.088496 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:19.088938 (+ 442us) server_negotiation.cc:299] Negotiation successful
0811 18:55:19.089051 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 18:55:19.090240 3324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:19.077898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:35851 (local address 127.27.180.130:57595)
0811 18:55:19.078326 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:19.078368 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:19.079577 (+ 1209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:19.080028 (+ 451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:19.080034 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:19.080359 (+ 325us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:19.080820 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:19.080833 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:19.081809 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:19.081813 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:19.082827 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:19.082838 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:19.083389 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:19.083392 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:19.083533 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:19.084044 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:19.084060 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:19.085675 (+ 1615us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:55:19.087394 (+ 1719us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:19.087399 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:19.087410 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:19.087648 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:19.087949 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:19.087951 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:19.087952 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:19.088070 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:19.088489 (+ 419us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:55:19.088494 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:55:19.088703 (+ 209us) client_negotiation.cc:769] Sending connection context
0811 18:55:19.088878 (+ 175us) client_negotiation.cc:240] Negotiation successful
0811 18:55:19.089224 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":102,"threads_started":1}
I20260811 18:55:19.091318 3322 heartbeater.cc:347] Connected to a master server at 127.27.180.190:35851
I20260811 18:55:19.091535 3322 heartbeater.cc:464] Registering TS with master...
I20260811 18:55:19.092007 3322 heartbeater.cc:511] Master 127.27.180.190:35851 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.27.180.131@KRBTEST.COM; defaulting to no policy
I20260811 18:55:19.093138 2638 ts_manager.cc:194] Registered new tserver with Master: 4d99ebced7114e3d992946c70212eda9 (127.27.180.130:41437)
I20260811 18:55:19.093757 2638 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.180.130@KRBTEST.COM'} at 127.27.180.130:57595
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.180.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.180.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.131.keytab.
Entry for principal kudu/127.27.180.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.131.keytab.
I20260811 18:55:19.109545 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474496157106-28370-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474496157106-28370-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474496157106-28370-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474496157106-28370-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.180.131:0
--local_ip_for_outbound_sockets=127.27.180.131
--webserver_interface=127.27.180.131
--webserver_port=0
--tserver_master_addrs=127.27.180.190:35851
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.131.keytab
--principal=kudu/127.27.180.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:41971
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:55:19.218094 3332 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:55:19.218354 3332 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:55:19.218395 3332 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:55:19.222554 3332 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:55:19.222615 3332 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:55:19.222671 3332 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.180.131
I20260811 18:55:19.227918 3332 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:41971
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474496157106-28370-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474496157106-28370-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.131.keytab
--principal=kudu/127.27.180.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.180.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474496157106-28370-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.180.131
--webserver_port=0
--tserver_master_addrs=127.27.180.190:35851
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3332
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.180.131
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474496157106-28370-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:55:19.228987 3332 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:55:19.230003 3332 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:55:19.239688 3341 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:19.239674 3340 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:19.239692 3346 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:55:19.239854 3332 server_base.cc:1034] running on GCE node
I20260811 18:55:19.240334 3332 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:55:19.240939 3332 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:55:19.242086 3332 hybrid_clock.cc:648] HybridClock initialized: now 1786474519242058 us; error 46 us; skew 500 ppm
Aug 11 18:55:19 dist-test-slave-2j7r krb5kdc[2553](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474519, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:55:19.244710 3332 init.cc:377] Logged in from keytab as kudu/127.27.180.131@KRBTEST.COM (short username kudu)
I20260811 18:55:19.245631 3332 webserver.cc:460] Webserver started at http://127.27.180.131:44707/ using document root <none> and password file <none>
I20260811 18:55:19.246171 3332 fs_manager.cc:359] Metadata directory not provided
I20260811 18:55:19.246230 3332 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:55:19.246430 3332 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:55:19.248481 3332 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474496157106-28370-0/minicluster-data/ts-2/data/instance:
uuid: "e88b50b04c1940859dae6e4c81a7e0ad"
format_stamp: "Formatted at 2026-08-11 18:55:19 on dist-test-slave-2j7r"
server_key: "0d0d6c7d6aa8ec21dea51578f1ed2eeb"
server_key_iv: "594f7c09523d811a2a36d4773d1b019c"
server_key_version: "encryptionkey@0"
I20260811 18:55:19.249012 3332 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474496157106-28370-0/minicluster-data/ts-2/wal/instance:
uuid: "e88b50b04c1940859dae6e4c81a7e0ad"
format_stamp: "Formatted at 2026-08-11 18:55:19 on dist-test-slave-2j7r"
server_key: "0d0d6c7d6aa8ec21dea51578f1ed2eeb"
server_key_iv: "594f7c09523d811a2a36d4773d1b019c"
server_key_version: "encryptionkey@0"
I20260811 18:55:19.252692 3332 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:55:19.255067 3360 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:19.256225 3332 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 18:55:19.256335 3332 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474496157106-28370-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474496157106-28370-0/minicluster-data/ts-2/wal
uuid: "e88b50b04c1940859dae6e4c81a7e0ad"
format_stamp: "Formatted at 2026-08-11 18:55:19 on dist-test-slave-2j7r"
server_key: "0d0d6c7d6aa8ec21dea51578f1ed2eeb"
server_key_iv: "594f7c09523d811a2a36d4773d1b019c"
server_key_version: "encryptionkey@0"
I20260811 18:55:19.256431 3332 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474496157106-28370-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474496157106-28370-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474496157106-28370-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:55:19.266882 3332 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:55:19.269928 3332 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:55:19.270102 3332 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:55:19.270668 3332 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:55:19.271478 3332 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:55:19.271525 3332 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:19.271577 3332 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:55:19.271606 3332 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:19.280341 3332 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.131:42099
I20260811 18:55:19.280368 3582 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.131:42099 every 8 connection(s)
I20260811 18:55:19.281235 3332 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474496157106-28370-0/minicluster-data/ts-2/data/info.pb
I20260811 18:55:19.285537 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 3332
I20260811 18:55:19.285657 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786474496157106-28370-0/minicluster-data/ts-2/wal/instance
I20260811 18:55:19.286001 28370 external_mini_cluster.cc:1428] Setting key 272746574082c60bf48f3f52dbc704c1
Aug 11 18:55:19 dist-test-slave-2j7r krb5kdc[2553](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474519, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.131@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM
I20260811 18:55:19.294808 2725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:19.283101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.131:39895 (local address 127.27.180.190:35851)
0811 18:55:19.283258 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:19.283263 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:19.284114 (+ 851us) server_negotiation.cc:407] Connection header received
0811 18:55:19.285664 (+ 1550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:19.285668 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:19.285730 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:19.285805 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:19.287213 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:19.287803 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:19.289014 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:19.289286 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:19.289376 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:19.291699 (+ 2323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:19.291702 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:19.291704 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:19.291740 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:19.292942 (+ 1202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:19.293484 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:19.293487 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:19.293488 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:19.293586 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:19.293872 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:19.293875 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:19.293876 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:19.294051 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:19.294129 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:19.294571 (+ 442us) server_negotiation.cc:299] Negotiation successful
0811 18:55:19.294673 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 18:55:19.295495 3586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:19.283486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:35851 (local address 127.27.180.131:39895)
0811 18:55:19.283957 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:19.284000 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:19.285419 (+ 1419us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:19.286006 (+ 587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:19.286015 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:19.286451 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:19.287063 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:19.287084 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:19.287929 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:19.287932 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:19.288888 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:19.288898 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:19.289438 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:19.289443 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:19.289621 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:19.290111 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:19.290126 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:19.291555 (+ 1429us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:55:19.293053 (+ 1498us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:19.293057 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:19.293070 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:19.293369 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:19.293666 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:19.293668 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:19.293669 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:19.293804 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:19.294129 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:55:19.294133 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:55:19.294345 (+ 212us) client_negotiation.cc:769] Sending connection context
0811 18:55:19.294492 (+ 147us) client_negotiation.cc:240] Negotiation successful
0811 18:55:19.294726 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":122,"threads_started":1}
I20260811 18:55:19.296561 3583 heartbeater.cc:347] Connected to a master server at 127.27.180.190:35851
I20260811 18:55:19.296811 3583 heartbeater.cc:464] Registering TS with master...
I20260811 18:55:19.297310 3583 heartbeater.cc:511] Master 127.27.180.190:35851 requested a full tablet report, sending...
I20260811 18:55:19.298316 2638 ts_manager.cc:194] Registered new tserver with Master: e88b50b04c1940859dae6e4c81a7e0ad (127.27.180.131:42099)
I20260811 18:55:19.299026 2638 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.180.131@KRBTEST.COM'} at 127.27.180.131:39895
I20260811 18:55:19.299906 28370 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 18:55:19 dist-test-slave-2j7r krb5kdc[2553](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474519, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 18:55:19 dist-test-slave-2j7r krb5kdc[2553](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474519, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.180.129@KRBTEST.COM
I20260811 18:55:19.326270 3594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:19.317045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41502 (local address 127.27.180.129:36991)
0811 18:55:19.317426 (+ 381us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:19.317437 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:19.317461 (+ 24us) server_negotiation.cc:407] Connection header received
0811 18:55:19.317513 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:19.317519 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:19.317699 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:19.317819 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:19.318432 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:19.319341 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:19.320283 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:19.320673 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:19.320754 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:19.323125 (+ 2371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:19.323135 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:19.323148 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:19.323173 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:19.324632 (+ 1459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:19.325048 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:19.325052 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:19.325053 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:19.325174 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:19.325423 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:19.325426 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:19.325427 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:19.325653 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:19.325814 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:19.325985 (+ 171us) server_negotiation.cc:299] Negotiation successful
0811 18:55:19.326111 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":259,"thread_start_us":156,"threads_started":1}
Aug 11 18:55:19 dist-test-slave-2j7r krb5kdc[2553](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474519, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM
I20260811 18:55:19.336658 2725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:19.329233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45972 (local address 127.27.180.190:35851)
0811 18:55:19.329373 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:19.329378 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:19.329392 (+ 14us) server_negotiation.cc:407] Connection header received
0811 18:55:19.329470 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:19.329472 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:19.329523 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:19.329584 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:19.330171 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:19.330735 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:19.331493 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:19.331714 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:19.331801 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:19.333963 (+ 2162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:19.333967 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:19.333970 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:19.333999 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:19.335129 (+ 1130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:19.335500 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:19.335503 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:19.335505 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:19.335585 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:19.335819 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:19.335823 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:19.335825 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:19.336052 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:19.336177 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:19.336386 (+ 209us) server_negotiation.cc:299] Negotiation successful
0811 18:55:19.336502 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
W20260811 18:55:19.337050 2638 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45972
I20260811 18:55:19.337730 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 2779
I20260811 18:55:19.355176 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 3039
I20260811 18:55:19.366176 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 3332
I20260811 18:55:19.377935 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 2580
2026-08-11T18:55:19Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2270 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474496157106-28370-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:55:19 dist-test-slave-2j7r krb5kdc[3603](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:55:19 dist-test-slave-2j7r krb5kdc[3603](info): set up 2 sockets
Aug 11 18:55:19 dist-test-slave-2j7r krb5kdc[3603](info): commencing operation
krb5kdc: starting...
W20260811 18:55:20.580426 28370 mini_kdc.cc:121] Time spent starting KDC: real 1.175s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474496157106-28370-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474496157106-28370-0/krb5kdc/test-user.keytab.
Aug 11 18:55:20 dist-test-slave-2j7r krb5kdc[3603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474520, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:55:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:55:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.180.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.180.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.180.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab.
Entry for principal kudu/127.27.180.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab.
I20260811 18:55:20.700434 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474496157106-28370-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474496157106-28370-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474496157106-28370-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.190:45645
--webserver_interface=127.27.180.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab
--principal=kudu/127.27.180.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:40691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.180.190:45645
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:55:20.807608 3626 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:55:20.807842 3626 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:55:20.807896 3626 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:55:20.811775 3626 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:55:20.811823 3626 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:55:20.811848 3626 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:55:20.811869 3626 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:55:20.811894 3626 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:55:20.816845 3626 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:40691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474496157106-28370-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474496157106-28370-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.180.190:45645
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab
--principal=kudu/127.27.180.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.190:45645
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.180.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3626
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474496157106-28370-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:55:20.817806 3626 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:55:20.818742 3626 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 18:55:20.826717 3626 server_base.cc:1034] running on GCE node
W20260811 18:55:20.826653 3646 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:20.826643 3657 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:20.826694 3642 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:55:20.827443 3626 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:55:20.828483 3626 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:55:20.829648 3626 hybrid_clock.cc:648] HybridClock initialized: now 1786474520829628 us; error 30 us; skew 500 ppm
Aug 11 18:55:20 dist-test-slave-2j7r krb5kdc[3603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474520, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:55:20.832376 3626 init.cc:377] Logged in from keytab as kudu/127.27.180.190@KRBTEST.COM (short username kudu)
I20260811 18:55:20.833076 3626 fs_manager.cc:359] Metadata directory not provided
I20260811 18:55:20.833195 3626 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:55:20.833411 3626 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:55:20.835433 3626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474496157106-28370-0/minicluster-data/master-0/data/instance:
uuid: "f4b773bf55ff4e398fcb775e672a743c"
format_stamp: "Formatted at 2026-08-11 18:55:20 on dist-test-slave-2j7r"
server_key: "ef4da44cad187036eff6795f9cd27e1d"
server_key_iv: "78ce190423c071b22c0557ef5c82cddf"
server_key_version: "encryptionkey@0"
I20260811 18:55:20.836033 3626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474496157106-28370-0/minicluster-data/master-0/wal/instance:
uuid: "f4b773bf55ff4e398fcb775e672a743c"
format_stamp: "Formatted at 2026-08-11 18:55:20 on dist-test-slave-2j7r"
server_key: "ef4da44cad187036eff6795f9cd27e1d"
server_key_iv: "78ce190423c071b22c0557ef5c82cddf"
server_key_version: "encryptionkey@0"
I20260811 18:55:20.839711 3626 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260811 18:55:20.842123 3670 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:20.843379 3626 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 18:55:20.843495 3626 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474496157106-28370-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474496157106-28370-0/minicluster-data/master-0/wal
uuid: "f4b773bf55ff4e398fcb775e672a743c"
format_stamp: "Formatted at 2026-08-11 18:55:20 on dist-test-slave-2j7r"
server_key: "ef4da44cad187036eff6795f9cd27e1d"
server_key_iv: "78ce190423c071b22c0557ef5c82cddf"
server_key_version: "encryptionkey@0"
I20260811 18:55:20.843592 3626 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474496157106-28370-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474496157106-28370-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474496157106-28370-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:55:20.870426 3626 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:55:20.873061 3626 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:55:20.873257 3626 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:55:20.879739 3626 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.190:45645
I20260811 18:55:20.880000 3772 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.190:45645 every 8 connection(s)
I20260811 18:55:20.880342 3626 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
I20260811 18:55:20.882932 3773 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:55:20.886569 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 3626
I20260811 18:55:20.886684 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474496157106-28370-0/minicluster-data/master-0/wal/instance
I20260811 18:55:20.886935 28370 external_mini_cluster.cc:1428] Setting key c5678e6687325a1cc5dc5375b6f85437
I20260811 18:55:20.888433 3773 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4b773bf55ff4e398fcb775e672a743c: Bootstrap starting.
I20260811 18:55:20.890810 3773 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f4b773bf55ff4e398fcb775e672a743c: Neither blocks nor log segments found. Creating new log.
I20260811 18:55:20.891494 3773 log.cc:824] T 00000000000000000000000000000000 P f4b773bf55ff4e398fcb775e672a743c: Log is configured to *not* fsync() on all Append() calls
I20260811 18:55:20.893532 3773 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4b773bf55ff4e398fcb775e672a743c: No bootstrap required, opened a new log
Aug 11 18:55:20 dist-test-slave-2j7r krb5kdc[3603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474520, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM
I20260811 18:55:20.896986 3773 raft_consensus.cc:348] T 00000000000000000000000000000000 P f4b773bf55ff4e398fcb775e672a743c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4b773bf55ff4e398fcb775e672a743c" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 45645 } }
I20260811 18:55:20.897276 3773 raft_consensus.cc:374] T 00000000000000000000000000000000 P f4b773bf55ff4e398fcb775e672a743c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:55:20.897341 3773 raft_consensus.cc:729] T 00000000000000000000000000000000 P f4b773bf55ff4e398fcb775e672a743c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4b773bf55ff4e398fcb775e672a743c, State: Initialized, Role: FOLLOWER
I20260811 18:55:20.897761 3773 consensus_queue.cc:260] T 00000000000000000000000000000000 P f4b773bf55ff4e398fcb775e672a743c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4b773bf55ff4e398fcb775e672a743c" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 45645 } }
I20260811 18:55:20.897895 3773 raft_consensus.cc:388] T 00000000000000000000000000000000 P f4b773bf55ff4e398fcb775e672a743c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:55:20.897957 3773 raft_consensus.cc:482] T 00000000000000000000000000000000 P f4b773bf55ff4e398fcb775e672a743c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:55:20.898052 3773 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f4b773bf55ff4e398fcb775e672a743c [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:55:20.899065 3773 raft_consensus.cc:504] T 00000000000000000000000000000000 P f4b773bf55ff4e398fcb775e672a743c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4b773bf55ff4e398fcb775e672a743c" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 45645 } }
I20260811 18:55:20.899389 3773 leader_election.cc:302] T 00000000000000000000000000000000 P f4b773bf55ff4e398fcb775e672a743c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f4b773bf55ff4e398fcb775e672a743c; no voters:
I20260811 18:55:20.899775 3773 leader_election.cc:288] T 00000000000000000000000000000000 P f4b773bf55ff4e398fcb775e672a743c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:55:20.899824 3785 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f4b773bf55ff4e398fcb775e672a743c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:55:20.900132 3785 raft_consensus.cc:686] T 00000000000000000000000000000000 P f4b773bf55ff4e398fcb775e672a743c [term 1 LEADER]: Becoming Leader. State: Replica: f4b773bf55ff4e398fcb775e672a743c, State: Running, Role: LEADER
I20260811 18:55:20.900491 3785 consensus_queue.cc:237] T 00000000000000000000000000000000 P f4b773bf55ff4e398fcb775e672a743c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4b773bf55ff4e398fcb775e672a743c" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 45645 } }
I20260811 18:55:20.901016 3773 sys_catalog.cc:564] T 00000000000000000000000000000000 P f4b773bf55ff4e398fcb775e672a743c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:55:20.901031 3783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:20.888134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59892 (local address 127.27.180.190:45645)
0811 18:55:20.888502 (+ 368us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:20.888531 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:20.888573 (+ 42us) server_negotiation.cc:407] Connection header received
0811 18:55:20.889239 (+ 666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:20.889255 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:20.889564 (+ 309us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:20.889825 (+ 261us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:20.890669 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:20.891665 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:20.892679 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:20.893179 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:20.893296 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:20.895942 (+ 2646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:20.895948 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:20.895961 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:20.895989 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:20.898036 (+ 2047us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:20.898492 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:20.898497 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:20.898501 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:20.898608 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:20.898872 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:20.898874 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:20.898875 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:20.899268 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:20.899401 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:20.899721 (+ 320us) server_negotiation.cc:299] Negotiation successful
0811 18:55:20.900049 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":238,"thread_start_us":101,"threads_started":1}
I20260811 18:55:20.902092 3789 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4b773bf55ff4e398fcb775e672a743c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f4b773bf55ff4e398fcb775e672a743c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4b773bf55ff4e398fcb775e672a743c" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 45645 } } }
I20260811 18:55:20.902288 3789 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4b773bf55ff4e398fcb775e672a743c [sys.catalog]: This master's current role is: LEADER
I20260811 18:55:20.902540 3789 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4b773bf55ff4e398fcb775e672a743c [sys.catalog]: SysCatalogTable state changed. Reason: New leader f4b773bf55ff4e398fcb775e672a743c. Latest consensus state: current_term: 1 leader_uuid: "f4b773bf55ff4e398fcb775e672a743c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4b773bf55ff4e398fcb775e672a743c" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 45645 } } }
I20260811 18:55:20.902627 3789 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4b773bf55ff4e398fcb775e672a743c [sys.catalog]: This master's current role is: LEADER
I20260811 18:55:20.903153 3805 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:55:20.906232 3805 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:55:20.911432 3805 catalog_manager.cc:1347] Generated new cluster ID: b925dc8f7b1e4a2cb867411c4bb0dd67
I20260811 18:55:20.911489 3805 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:55:20.925546 3805 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:55:20.926784 3805 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:55:20.932407 3805 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f4b773bf55ff4e398fcb775e672a743c: Generated new TSK 0
I20260811 18:55:20.933180 3805 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.180.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.180.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.180.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.129.keytab.
Entry for principal kudu/127.27.180.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.129.keytab.
I20260811 18:55:20.964969 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474496157106-28370-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474496157106-28370-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474496157106-28370-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474496157106-28370-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.180.129:0
--local_ip_for_outbound_sockets=127.27.180.129
--webserver_interface=127.27.180.129
--webserver_port=0
--tserver_master_addrs=127.27.180.190:45645
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.129.keytab
--principal=kudu/127.27.180.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:40691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:55:21.075528 3838 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:55:21.075757 3838 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:55:21.075798 3838 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:55:21.079689 3838 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:55:21.079739 3838 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:55:21.079818 3838 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.180.129
I20260811 18:55:21.084782 3838 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:40691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474496157106-28370-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474496157106-28370-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.129.keytab
--principal=kudu/127.27.180.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.180.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474496157106-28370-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.180.129
--webserver_port=0
--tserver_master_addrs=127.27.180.190:45645
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3838
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.180.129
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474496157106-28370-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:55:21.085747 3838 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:55:21.086717 3838 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:55:21.095614 3847 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:21.095590 3855 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:55:21.095594 3838 server_base.cc:1034] running on GCE node
W20260811 18:55:21.095631 3850 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:55:21.096246 3838 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:55:21.096843 3838 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:55:21.097998 3838 hybrid_clock.cc:648] HybridClock initialized: now 1786474521097964 us; error 45 us; skew 500 ppm
Aug 11 18:55:21 dist-test-slave-2j7r krb5kdc[3603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474521, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:55:21.100670 3838 init.cc:377] Logged in from keytab as kudu/127.27.180.129@KRBTEST.COM (short username kudu)
I20260811 18:55:21.101410 3838 fs_manager.cc:359] Metadata directory not provided
I20260811 18:55:21.101483 3838 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:55:21.101683 3838 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:55:21.103685 3838 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474496157106-28370-0/minicluster-data/ts-0/data/instance:
uuid: "63513ded32fa459ca84ed7478e36cfbe"
format_stamp: "Formatted at 2026-08-11 18:55:21 on dist-test-slave-2j7r"
server_key: "a598a10ebd64033f5d1336acf69d6db8"
server_key_iv: "b7d9fc8a427e45b0f1dce40e0ca0f5cd"
server_key_version: "encryptionkey@0"
I20260811 18:55:21.104233 3838 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474496157106-28370-0/minicluster-data/ts-0/wal/instance:
uuid: "63513ded32fa459ca84ed7478e36cfbe"
format_stamp: "Formatted at 2026-08-11 18:55:21 on dist-test-slave-2j7r"
server_key: "a598a10ebd64033f5d1336acf69d6db8"
server_key_iv: "b7d9fc8a427e45b0f1dce40e0ca0f5cd"
server_key_version: "encryptionkey@0"
I20260811 18:55:21.107784 3838 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260811 18:55:21.110141 3866 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:21.111372 3838 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 18:55:21.111490 3838 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474496157106-28370-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474496157106-28370-0/minicluster-data/ts-0/wal
uuid: "63513ded32fa459ca84ed7478e36cfbe"
format_stamp: "Formatted at 2026-08-11 18:55:21 on dist-test-slave-2j7r"
server_key: "a598a10ebd64033f5d1336acf69d6db8"
server_key_iv: "b7d9fc8a427e45b0f1dce40e0ca0f5cd"
server_key_version: "encryptionkey@0"
I20260811 18:55:21.111580 3838 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474496157106-28370-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474496157106-28370-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474496157106-28370-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:55:21.125226 3838 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:55:21.127830 3838 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:55:21.127990 3838 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:55:21.128525 3838 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:55:21.129319 3838 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:55:21.129366 3838 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:21.129419 3838 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:55:21.129444 3838 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:21.137758 3838 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.129:43763
I20260811 18:55:21.137781 4101 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.129:43763 every 8 connection(s)
I20260811 18:55:21.138321 3838 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474496157106-28370-0/minicluster-data/ts-0/data/info.pb
I20260811 18:55:21.141327 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 3838
I20260811 18:55:21.141453 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474496157106-28370-0/minicluster-data/ts-0/wal/instance
I20260811 18:55:21.141706 28370 external_mini_cluster.cc:1428] Setting key 8fb28b24974e291577391c86dcb74792
Aug 11 18:55:21 dist-test-slave-2j7r krb5kdc[3603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474521, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.129@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.27.180.130@KRBTEST.COM; defaulting to no policy
I20260811 18:55:21.152781 3783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:21.139999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.129:55421 (local address 127.27.180.190:45645)
0811 18:55:21.140123 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:21.140127 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:21.140938 (+ 811us) server_negotiation.cc:407] Connection header received
0811 18:55:21.142544 (+ 1606us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:21.142548 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:21.142593 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:21.142666 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:21.143757 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:21.144278 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:21.145769 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:21.146074 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:21.146242 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:21.148891 (+ 2649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:21.148894 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:21.148896 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:21.148922 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:21.150431 (+ 1509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:21.151043 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:21.151046 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:21.151047 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:21.151135 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:21.151587 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:21.151591 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:21.151592 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:21.151790 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:21.151881 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:21.152537 (+ 656us) server_negotiation.cc:299] Negotiation successful
0811 18:55:21.152647 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 18:55:21.153596 4108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:21.140330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:45645 (local address 127.27.180.129:55421)
0811 18:55:21.140792 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:21.140834 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:21.142315 (+ 1481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:21.142822 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:21.142828 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:21.143169 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:21.143616 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:21.143628 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:21.144395 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:21.144398 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:21.145620 (+ 1222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:21.145632 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:21.146251 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:21.146255 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:21.146402 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:21.146950 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:21.146966 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:21.148713 (+ 1747us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:55:21.150551 (+ 1838us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:21.150558 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:21.150573 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:21.150934 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:21.151263 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:21.151267 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:21.151269 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:21.151452 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:21.151909 (+ 457us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:55:21.151917 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:55:21.152215 (+ 298us) client_negotiation.cc:769] Sending connection context
0811 18:55:21.152444 (+ 229us) client_negotiation.cc:240] Negotiation successful
0811 18:55:21.152757 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":107,"threads_started":1}
I20260811 18:55:21.154719 4103 heartbeater.cc:347] Connected to a master server at 127.27.180.190:45645
I20260811 18:55:21.154992 4103 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.180.130@KRBTEST.COM" created.
I20260811 18:55:21.155463 4103 heartbeater.cc:511] Master 127.27.180.190:45645 requested a full tablet report, sending...
I20260811 18:55:21.156955 3701 ts_manager.cc:194] Registered new tserver with Master: 63513ded32fa459ca84ed7478e36cfbe (127.27.180.129:43763)
I20260811 18:55:21.158084 3701 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.180.129@KRBTEST.COM'} at 127.27.180.129:55421
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.180.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.130.keytab.
Entry for principal kudu/127.27.180.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.130.keytab.
I20260811 18:55:21.172037 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474496157106-28370-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474496157106-28370-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474496157106-28370-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474496157106-28370-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.180.130:0
--local_ip_for_outbound_sockets=127.27.180.130
--webserver_interface=127.27.180.130
--webserver_port=0
--tserver_master_addrs=127.27.180.190:45645
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.130.keytab
--principal=kudu/127.27.180.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:40691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:55:21.281512 4114 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:55:21.281750 4114 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:55:21.281807 4114 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:55:21.285697 4114 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:55:21.285748 4114 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:55:21.285820 4114 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.180.130
I20260811 18:55:21.290892 4114 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:40691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474496157106-28370-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474496157106-28370-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.130.keytab
--principal=kudu/127.27.180.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.180.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474496157106-28370-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.180.130
--webserver_port=0
--tserver_master_addrs=127.27.180.190:45645
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4114
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.180.130
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474496157106-28370-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:55:21.291826 4114 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:55:21.292850 4114 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:55:21.301833 4129 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:21.301827 4128 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:21.301836 4131 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:55:21.302235 4114 server_base.cc:1034] running on GCE node
I20260811 18:55:21.302615 4114 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:55:21.303215 4114 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:55:21.304395 4114 hybrid_clock.cc:648] HybridClock initialized: now 1786474521304375 us; error 34 us; skew 500 ppm
Aug 11 18:55:21 dist-test-slave-2j7r krb5kdc[3603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474521, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:55:21.307001 4114 init.cc:377] Logged in from keytab as kudu/127.27.180.130@KRBTEST.COM (short username kudu)
I20260811 18:55:21.307739 4114 fs_manager.cc:359] Metadata directory not provided
I20260811 18:55:21.307804 4114 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:55:21.308007 4114 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:55:21.309983 4114 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474496157106-28370-0/minicluster-data/ts-1/data/instance:
uuid: "34f73195691f44ccb727078f5f420efc"
format_stamp: "Formatted at 2026-08-11 18:55:21 on dist-test-slave-2j7r"
server_key: "e812985dad2766083e8ac84dd659dd19"
server_key_iv: "7f80ffb030c60f7b587b8b282c4a3e55"
server_key_version: "encryptionkey@0"
I20260811 18:55:21.310513 4114 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474496157106-28370-0/minicluster-data/ts-1/wal/instance:
uuid: "34f73195691f44ccb727078f5f420efc"
format_stamp: "Formatted at 2026-08-11 18:55:21 on dist-test-slave-2j7r"
server_key: "e812985dad2766083e8ac84dd659dd19"
server_key_iv: "7f80ffb030c60f7b587b8b282c4a3e55"
server_key_version: "encryptionkey@0"
I20260811 18:55:21.313939 4114 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260811 18:55:21.316174 4143 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:21.317365 4114 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 18:55:21.317479 4114 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474496157106-28370-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474496157106-28370-0/minicluster-data/ts-1/wal
uuid: "34f73195691f44ccb727078f5f420efc"
format_stamp: "Formatted at 2026-08-11 18:55:21 on dist-test-slave-2j7r"
server_key: "e812985dad2766083e8ac84dd659dd19"
server_key_iv: "7f80ffb030c60f7b587b8b282c4a3e55"
server_key_version: "encryptionkey@0"
I20260811 18:55:21.317600 4114 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474496157106-28370-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474496157106-28370-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474496157106-28370-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:55:21.341470 4114 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:55:21.344336 4114 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:55:21.344502 4114 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:55:21.345042 4114 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:55:21.345855 4114 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:55:21.345903 4114 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:21.345969 4114 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:55:21.345995 4114 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:21.354507 4114 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.130:43603
I20260811 18:55:21.354532 4362 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.130:43603 every 8 connection(s)
I20260811 18:55:21.355019 4114 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474496157106-28370-0/minicluster-data/ts-1/data/info.pb
I20260811 18:55:21.358273 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 4114
I20260811 18:55:21.358388 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474496157106-28370-0/minicluster-data/ts-1/wal/instance
I20260811 18:55:21.358685 28370 external_mini_cluster.cc:1428] Setting key c238b277870d4c2214a0e267fc73f733
Aug 11 18:55:21 dist-test-slave-2j7r krb5kdc[3603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474521, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.130@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM
I20260811 18:55:21.369016 3783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:21.356593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.130:33669 (local address 127.27.180.190:45645)
0811 18:55:21.356727 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:21.356730 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:21.357538 (+ 808us) server_negotiation.cc:407] Connection header received
0811 18:55:21.359075 (+ 1537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:21.359078 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:21.359137 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:21.359229 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:21.360515 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:21.361025 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:21.362457 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:21.362686 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:21.362776 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:21.365364 (+ 2588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:21.365368 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:21.365370 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:21.365395 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:21.366744 (+ 1349us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:21.367361 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:21.367365 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:21.367366 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:21.367455 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:21.367870 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:21.367873 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:21.367874 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:21.368087 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:21.368184 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:21.368780 (+ 596us) server_negotiation.cc:299] Negotiation successful
0811 18:55:21.368892 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
WARNING: no policy specified for kudu/127.27.180.131@KRBTEST.COM; defaulting to no policy
I20260811 18:55:21.369910 4365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:21.356943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:45645 (local address 127.27.180.130:33669)
0811 18:55:21.357401 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:21.357443 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:21.358852 (+ 1409us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:21.359399 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:21.359408 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:21.359753 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:21.360363 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:21.360384 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:21.361158 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:21.361161 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:21.362341 (+ 1180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:21.362352 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:21.362829 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:21.362833 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:21.362983 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:21.363491 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:21.363508 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:21.365197 (+ 1689us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:55:21.366866 (+ 1669us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:21.366872 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:21.366886 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:21.367221 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:21.367543 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:21.367547 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:21.367549 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:21.367728 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:21.368202 (+ 474us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:55:21.368209 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:55:21.368524 (+ 315us) client_negotiation.cc:769] Sending connection context
0811 18:55:21.368728 (+ 204us) client_negotiation.cc:240] Negotiation successful
0811 18:55:21.369045 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":131,"threads_started":1}
I20260811 18:55:21.371076 4363 heartbeater.cc:347] Connected to a master server at 127.27.180.190:45645
I20260811 18:55:21.371302 4363 heartbeater.cc:464] Registering TS with master...
I20260811 18:55:21.371744 4363 heartbeater.cc:511] Master 127.27.180.190:45645 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.180.131@KRBTEST.COM" created.
I20260811 18:55:21.372903 3701 ts_manager.cc:194] Registered new tserver with Master: 34f73195691f44ccb727078f5f420efc (127.27.180.130:43603)
I20260811 18:55:21.373667 3701 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.180.130@KRBTEST.COM'} at 127.27.180.130:33669
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.180.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.131.keytab.
Entry for principal kudu/127.27.180.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.131.keytab.
I20260811 18:55:21.386564 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474496157106-28370-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474496157106-28370-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474496157106-28370-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474496157106-28370-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.180.131:0
--local_ip_for_outbound_sockets=127.27.180.131
--webserver_interface=127.27.180.131
--webserver_port=0
--tserver_master_addrs=127.27.180.190:45645
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.131.keytab
--principal=kudu/127.27.180.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:40691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:55:21.497711 4368 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:55:21.497941 4368 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:55:21.497985 4368 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:55:21.501947 4368 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:55:21.501999 4368 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:55:21.502061 4368 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.180.131
I20260811 18:55:21.507547 4368 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:40691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474496157106-28370-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474496157106-28370-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.131.keytab
--principal=kudu/127.27.180.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.180.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474496157106-28370-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.180.131
--webserver_port=0
--tserver_master_addrs=127.27.180.190:45645
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4368
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.180.131
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474496157106-28370-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:55:21.508627 4368 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:55:21.509724 4368 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:55:21.518950 4379 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:55:21.519073 4368 server_base.cc:1034] running on GCE node
W20260811 18:55:21.518931 4382 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:21.518935 4380 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:55:21.519577 4368 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:55:21.520146 4368 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:55:21.521298 4368 hybrid_clock.cc:648] HybridClock initialized: now 1786474521521284 us; error 25 us; skew 500 ppm
Aug 11 18:55:21 dist-test-slave-2j7r krb5kdc[3603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474521, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:55:21.523895 4368 init.cc:377] Logged in from keytab as kudu/127.27.180.131@KRBTEST.COM (short username kudu)
I20260811 18:55:21.524559 4368 fs_manager.cc:359] Metadata directory not provided
I20260811 18:55:21.524626 4368 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:55:21.524806 4368 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:55:21.526758 4368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474496157106-28370-0/minicluster-data/ts-2/data/instance:
uuid: "3a5c80e69d0f44babcc7a6e18a2b5429"
format_stamp: "Formatted at 2026-08-11 18:55:21 on dist-test-slave-2j7r"
server_key: "42cdfb6d770f3e02e7872802295687a6"
server_key_iv: "c82152f5cede46a6e641fe4ce27500f2"
server_key_version: "encryptionkey@0"
I20260811 18:55:21.527277 4368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474496157106-28370-0/minicluster-data/ts-2/wal/instance:
uuid: "3a5c80e69d0f44babcc7a6e18a2b5429"
format_stamp: "Formatted at 2026-08-11 18:55:21 on dist-test-slave-2j7r"
server_key: "42cdfb6d770f3e02e7872802295687a6"
server_key_iv: "c82152f5cede46a6e641fe4ce27500f2"
server_key_version: "encryptionkey@0"
I20260811 18:55:21.530680 4368 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 18:55:21.532892 4393 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:21.534021 4368 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 18:55:21.534123 4368 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474496157106-28370-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474496157106-28370-0/minicluster-data/ts-2/wal
uuid: "3a5c80e69d0f44babcc7a6e18a2b5429"
format_stamp: "Formatted at 2026-08-11 18:55:21 on dist-test-slave-2j7r"
server_key: "42cdfb6d770f3e02e7872802295687a6"
server_key_iv: "c82152f5cede46a6e641fe4ce27500f2"
server_key_version: "encryptionkey@0"
I20260811 18:55:21.534211 4368 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474496157106-28370-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474496157106-28370-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474496157106-28370-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:55:21.552680 4368 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:55:21.555365 4368 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:55:21.555531 4368 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:55:21.556067 4368 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:55:21.556861 4368 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:55:21.556907 4368 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:21.556958 4368 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:55:21.556988 4368 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:21.565186 4368 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.131:43743
I20260811 18:55:21.565222 4623 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.131:43743 every 8 connection(s)
I20260811 18:55:21.565726 4368 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474496157106-28370-0/minicluster-data/ts-2/data/info.pb
I20260811 18:55:21.572525 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 4368
I20260811 18:55:21.572652 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474496157106-28370-0/minicluster-data/ts-2/wal/instance
I20260811 18:55:21.572962 28370 external_mini_cluster.cc:1428] Setting key 68e7d1475d251428cdad0228037cad8c
Aug 11 18:55:21 dist-test-slave-2j7r krb5kdc[3603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474521, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.131@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM
I20260811 18:55:21.578471 3783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:21.567319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.131:53417 (local address 127.27.180.190:45645)
0811 18:55:21.567483 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:21.567488 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:21.568477 (+ 989us) server_negotiation.cc:407] Connection header received
0811 18:55:21.569275 (+ 798us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:21.569278 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:21.569327 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:21.569413 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:21.570511 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:21.571063 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:21.572167 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:21.572540 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:21.572667 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:21.575244 (+ 2577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:21.575248 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:21.575250 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:21.575278 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:21.576511 (+ 1233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:21.577011 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:21.577015 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:21.577017 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:21.577136 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:21.577463 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:21.577467 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:21.577469 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:21.577650 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:21.577746 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:21.578190 (+ 444us) server_negotiation.cc:299] Negotiation successful
0811 18:55:21.578313 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 18:55:21.579174 4635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:21.567666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:45645 (local address 127.27.180.131:53417)
0811 18:55:21.568341 (+ 675us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:21.568379 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:21.569107 (+ 728us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:21.569554 (+ 447us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:21.569559 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:21.569906 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:21.570351 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:21.570364 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:21.571180 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:21.571183 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:21.572041 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:21.572050 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:21.572716 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:21.572721 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:21.572872 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:21.573398 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:21.573417 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:21.575085 (+ 1668us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:55:21.576631 (+ 1546us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:21.576637 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:21.576651 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:21.576908 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:21.577238 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:21.577241 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:21.577243 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:21.577376 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:21.577748 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:55:21.577755 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:55:21.577973 (+ 218us) client_negotiation.cc:769] Sending connection context
0811 18:55:21.578130 (+ 157us) client_negotiation.cc:240] Negotiation successful
0811 18:55:21.578434 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":122,"threads_started":1}
I20260811 18:55:21.580168 4627 heartbeater.cc:347] Connected to a master server at 127.27.180.190:45645
I20260811 18:55:21.580365 4627 heartbeater.cc:464] Registering TS with master...
I20260811 18:55:21.580741 4627 heartbeater.cc:511] Master 127.27.180.190:45645 requested a full tablet report, sending...
I20260811 18:55:21.581727 3701 ts_manager.cc:194] Registered new tserver with Master: 3a5c80e69d0f44babcc7a6e18a2b5429 (127.27.180.131:43743)
I20260811 18:55:21.582309 3701 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.180.131@KRBTEST.COM'} at 127.27.180.131:53417
I20260811 18:55:21.587031 28370 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 18:55:21.595394 3783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:21.588866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59896 (local address 127.27.180.190:45645)
0811 18:55:21.588992 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:21.588996 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:21.589164 (+ 168us) server_negotiation.cc:407] Connection header received
0811 18:55:21.589302 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:21.589304 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:21.589342 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:21.589404 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:21.590133 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:21.590604 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:21.591412 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:21.591612 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:21.591677 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:21.592832 (+ 1155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:21.592835 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:21.592837 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:21.592858 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:21.594127 (+ 1269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:21.594510 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:21.594513 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:21.594514 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:21.594576 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:21.594755 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:21.594757 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:21.594758 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:21.594950 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:21.595044 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:21.595191 (+ 147us) server_negotiation.cc:299] Negotiation successful
0811 18:55:21.595273 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 18:55:21.597977 3701 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59896:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:55:21.600889 3701 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:55:21.614283 4654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:21.608766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.129:43763 (local address 127.0.0.1:55774)
0811 18:55:21.609228 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:21.609274 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:21.609392 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:21.610012 (+ 620us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:21.610018 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:21.610038 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:55:21.610164 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:21.610172 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:21.611571 (+ 1399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:21.611575 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:21.612697 (+ 1122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:21.612708 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:21.613752 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:21.613756 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:21.613869 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:21.613909 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 18:55:21.614019 (+ 110us) client_negotiation.cc:240] Negotiation successful
0811 18:55:21.614134 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"mutex_wait_us":41,"thread_start_us":74,"threads_started":1}
I20260811 18:55:21.614316 4655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:21.608927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55774 (local address 127.27.180.129:43763)
0811 18:55:21.609525 (+ 598us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:21.609537 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:21.609563 (+ 26us) server_negotiation.cc:407] Connection header received
0811 18:55:21.609641 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:21.609646 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:21.609863 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:21.610031 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:55:21.610493 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:21.611457 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:21.612834 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:21.613639 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:21.613751 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:21.613878 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:21.614051 (+ 173us) server_negotiation.cc:299] Negotiation successful
0811 18:55:21.614151 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":495,"thread_start_us":126,"threads_started":1}
I20260811 18:55:21.614802 4658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:21.608857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.130:43603 (local address 127.0.0.1:46574)
0811 18:55:21.609934 (+ 1077us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:21.609947 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:21.610387 (+ 440us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:21.610781 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:21.610783 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:21.610799 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:55:21.610885 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:21.610891 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:21.612201 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:21.612204 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:21.613307 (+ 1103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:21.613317 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:21.614451 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:21.614454 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:21.614539 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:21.614561 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 18:55:21.614629 (+ 68us) client_negotiation.cc:240] Negotiation successful
0811 18:55:21.614687 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1019,"thread_start_us":82,"threads_started":1}
I20260811 18:55:21.614907 4656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:21.609344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46574 (local address 127.27.180.130:43603)
0811 18:55:21.609791 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:21.609802 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:21.610323 (+ 521us) server_negotiation.cc:407] Connection header received
0811 18:55:21.610509 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:21.610513 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:21.610649 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:21.610779 (+ 130us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:55:21.610991 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:21.612052 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:21.613425 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:21.614153 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:21.614271 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:21.614387 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:21.614672 (+ 285us) server_negotiation.cc:299] Negotiation successful
0811 18:55:21.614765 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":125,"threads_started":1}
I20260811 18:55:21.616492 4653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:21.609486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.131:43743 (local address 127.0.0.1:47966)
0811 18:55:21.610261 (+ 775us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:21.610274 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:21.610373 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:21.610961 (+ 588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:21.610963 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:21.610981 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:55:21.611075 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:21.611080 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:21.612412 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:21.612416 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:21.613495 (+ 1079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:21.613506 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:21.615399 (+ 1893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:21.615403 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:21.615488 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:21.615511 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 18:55:21.616171 (+ 660us) client_negotiation.cc:240] Negotiation successful
0811 18:55:21.616258 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":718,"mutex_wait_us":26,"thread_start_us":89,"threads_started":1}
I20260811 18:55:21.616498 4660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:21.609862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47966 (local address 127.27.180.131:43743)
0811 18:55:21.610276 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:21.610289 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:21.610366 (+ 77us) server_negotiation.cc:407] Connection header received
0811 18:55:21.610492 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:21.610498 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:21.610675 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:21.610816 (+ 141us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:55:21.611194 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:21.612273 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:21.613621 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:21.614464 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:21.615390 (+ 926us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:21.615503 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:21.616241 (+ 738us) server_negotiation.cc:299] Negotiation successful
0811 18:55:21.616348 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":91,"threads_started":1}
I20260811 18:55:21.617509 3976 tablet_service.cc:1467] Processing CreateTablet for tablet 230802eaaac846d59c781eb86c050d10 (DEFAULT_TABLE table=test-table [id=e498b6c50ff941368597e2850978d279]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:55:21.617509 4239 tablet_service.cc:1467] Processing CreateTablet for tablet 230802eaaac846d59c781eb86c050d10 (DEFAULT_TABLE table=test-table [id=e498b6c50ff941368597e2850978d279]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:55:21.618626 3976 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 230802eaaac846d59c781eb86c050d10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:55:21.618625 4239 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 230802eaaac846d59c781eb86c050d10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:55:21.619149 4485 tablet_service.cc:1467] Processing CreateTablet for tablet 230802eaaac846d59c781eb86c050d10 (DEFAULT_TABLE table=test-table [id=e498b6c50ff941368597e2850978d279]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:55:21.620100 4485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 230802eaaac846d59c781eb86c050d10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:55:21.625895 4662 tablet_bootstrap.cc:492] T 230802eaaac846d59c781eb86c050d10 P 63513ded32fa459ca84ed7478e36cfbe: Bootstrap starting.
I20260811 18:55:21.627141 4665 tablet_bootstrap.cc:492] T 230802eaaac846d59c781eb86c050d10 P 3a5c80e69d0f44babcc7a6e18a2b5429: Bootstrap starting.
I20260811 18:55:21.627343 4664 tablet_bootstrap.cc:492] T 230802eaaac846d59c781eb86c050d10 P 34f73195691f44ccb727078f5f420efc: Bootstrap starting.
I20260811 18:55:21.628746 4662 tablet_bootstrap.cc:654] T 230802eaaac846d59c781eb86c050d10 P 63513ded32fa459ca84ed7478e36cfbe: Neither blocks nor log segments found. Creating new log.
I20260811 18:55:21.629020 4664 tablet_bootstrap.cc:654] T 230802eaaac846d59c781eb86c050d10 P 34f73195691f44ccb727078f5f420efc: Neither blocks nor log segments found. Creating new log.
I20260811 18:55:21.629310 4665 tablet_bootstrap.cc:654] T 230802eaaac846d59c781eb86c050d10 P 3a5c80e69d0f44babcc7a6e18a2b5429: Neither blocks nor log segments found. Creating new log.
I20260811 18:55:21.630074 4664 log.cc:824] T 230802eaaac846d59c781eb86c050d10 P 34f73195691f44ccb727078f5f420efc: Log is configured to *not* fsync() on all Append() calls
I20260811 18:55:21.630160 4662 log.cc:824] T 230802eaaac846d59c781eb86c050d10 P 63513ded32fa459ca84ed7478e36cfbe: Log is configured to *not* fsync() on all Append() calls
I20260811 18:55:21.630511 4665 log.cc:824] T 230802eaaac846d59c781eb86c050d10 P 3a5c80e69d0f44babcc7a6e18a2b5429: Log is configured to *not* fsync() on all Append() calls
I20260811 18:55:21.631803 4664 tablet_bootstrap.cc:492] T 230802eaaac846d59c781eb86c050d10 P 34f73195691f44ccb727078f5f420efc: No bootstrap required, opened a new log
I20260811 18:55:21.631976 4664 ts_tablet_manager.cc:1397] T 230802eaaac846d59c781eb86c050d10 P 34f73195691f44ccb727078f5f420efc: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260811 18:55:21.632160 4665 tablet_bootstrap.cc:492] T 230802eaaac846d59c781eb86c050d10 P 3a5c80e69d0f44babcc7a6e18a2b5429: No bootstrap required, opened a new log
I20260811 18:55:21.632339 4665 ts_tablet_manager.cc:1397] T 230802eaaac846d59c781eb86c050d10 P 3a5c80e69d0f44babcc7a6e18a2b5429: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 18:55:21.634459 4662 tablet_bootstrap.cc:492] T 230802eaaac846d59c781eb86c050d10 P 63513ded32fa459ca84ed7478e36cfbe: No bootstrap required, opened a new log
I20260811 18:55:21.634657 4662 ts_tablet_manager.cc:1397] T 230802eaaac846d59c781eb86c050d10 P 63513ded32fa459ca84ed7478e36cfbe: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.003s
I20260811 18:55:21.635067 4664 raft_consensus.cc:348] T 230802eaaac846d59c781eb86c050d10 P 34f73195691f44ccb727078f5f420efc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63513ded32fa459ca84ed7478e36cfbe" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 43763 } } peers { permanent_uuid: "34f73195691f44ccb727078f5f420efc" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 43603 } } peers { permanent_uuid: "3a5c80e69d0f44babcc7a6e18a2b5429" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 43743 } }
I20260811 18:55:21.635296 4664 raft_consensus.cc:374] T 230802eaaac846d59c781eb86c050d10 P 34f73195691f44ccb727078f5f420efc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:55:21.635358 4664 raft_consensus.cc:729] T 230802eaaac846d59c781eb86c050d10 P 34f73195691f44ccb727078f5f420efc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34f73195691f44ccb727078f5f420efc, State: Initialized, Role: FOLLOWER
I20260811 18:55:21.635761 4664 consensus_queue.cc:260] T 230802eaaac846d59c781eb86c050d10 P 34f73195691f44ccb727078f5f420efc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63513ded32fa459ca84ed7478e36cfbe" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 43763 } } peers { permanent_uuid: "34f73195691f44ccb727078f5f420efc" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 43603 } } peers { permanent_uuid: "3a5c80e69d0f44babcc7a6e18a2b5429" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 43743 } }
I20260811 18:55:21.636102 4665 raft_consensus.cc:348] T 230802eaaac846d59c781eb86c050d10 P 3a5c80e69d0f44babcc7a6e18a2b5429 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63513ded32fa459ca84ed7478e36cfbe" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 43763 } } peers { permanent_uuid: "34f73195691f44ccb727078f5f420efc" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 43603 } } peers { permanent_uuid: "3a5c80e69d0f44babcc7a6e18a2b5429" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 43743 } }
I20260811 18:55:21.636389 4363 heartbeater.cc:503] Master 127.27.180.190:45645 was elected leader, sending a full tablet report...
I20260811 18:55:21.636404 4665 raft_consensus.cc:374] T 230802eaaac846d59c781eb86c050d10 P 3a5c80e69d0f44babcc7a6e18a2b5429 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:55:21.636461 4665 raft_consensus.cc:729] T 230802eaaac846d59c781eb86c050d10 P 3a5c80e69d0f44babcc7a6e18a2b5429 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a5c80e69d0f44babcc7a6e18a2b5429, State: Initialized, Role: FOLLOWER
I20260811 18:55:21.636708 4664 ts_tablet_manager.cc:1428] T 230802eaaac846d59c781eb86c050d10 P 34f73195691f44ccb727078f5f420efc: Time spent starting tablet: real 0.005s user 0.001s sys 0.004s
I20260811 18:55:21.636924 4665 consensus_queue.cc:260] T 230802eaaac846d59c781eb86c050d10 P 3a5c80e69d0f44babcc7a6e18a2b5429 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63513ded32fa459ca84ed7478e36cfbe" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 43763 } } peers { permanent_uuid: "34f73195691f44ccb727078f5f420efc" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 43603 } } peers { permanent_uuid: "3a5c80e69d0f44babcc7a6e18a2b5429" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 43743 } }
I20260811 18:55:21.637691 4627 heartbeater.cc:503] Master 127.27.180.190:45645 was elected leader, sending a full tablet report...
I20260811 18:55:21.638072 4665 ts_tablet_manager.cc:1428] T 230802eaaac846d59c781eb86c050d10 P 3a5c80e69d0f44babcc7a6e18a2b5429: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 18:55:21.639571 4662 raft_consensus.cc:348] T 230802eaaac846d59c781eb86c050d10 P 63513ded32fa459ca84ed7478e36cfbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63513ded32fa459ca84ed7478e36cfbe" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 43763 } } peers { permanent_uuid: "34f73195691f44ccb727078f5f420efc" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 43603 } } peers { permanent_uuid: "3a5c80e69d0f44babcc7a6e18a2b5429" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 43743 } }
I20260811 18:55:21.639832 4662 raft_consensus.cc:374] T 230802eaaac846d59c781eb86c050d10 P 63513ded32fa459ca84ed7478e36cfbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:55:21.639906 4662 raft_consensus.cc:729] T 230802eaaac846d59c781eb86c050d10 P 63513ded32fa459ca84ed7478e36cfbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63513ded32fa459ca84ed7478e36cfbe, State: Initialized, Role: FOLLOWER
I20260811 18:55:21.640372 4662 consensus_queue.cc:260] T 230802eaaac846d59c781eb86c050d10 P 63513ded32fa459ca84ed7478e36cfbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63513ded32fa459ca84ed7478e36cfbe" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 43763 } } peers { permanent_uuid: "34f73195691f44ccb727078f5f420efc" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 43603 } } peers { permanent_uuid: "3a5c80e69d0f44babcc7a6e18a2b5429" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 43743 } }
I20260811 18:55:21.641876 4662 ts_tablet_manager.cc:1428] T 230802eaaac846d59c781eb86c050d10 P 63513ded32fa459ca84ed7478e36cfbe: Time spent starting tablet: real 0.007s user 0.000s sys 0.006s
I20260811 18:55:21.642052 4103 heartbeater.cc:503] Master 127.27.180.190:45645 was elected leader, sending a full tablet report...
I20260811 18:55:21.764928 4670 raft_consensus.cc:482] T 230802eaaac846d59c781eb86c050d10 P 34f73195691f44ccb727078f5f420efc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:55:21.765163 4670 raft_consensus.cc:504] T 230802eaaac846d59c781eb86c050d10 P 34f73195691f44ccb727078f5f420efc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63513ded32fa459ca84ed7478e36cfbe" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 43763 } } peers { permanent_uuid: "34f73195691f44ccb727078f5f420efc" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 43603 } } peers { permanent_uuid: "3a5c80e69d0f44babcc7a6e18a2b5429" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 43743 } }
I20260811 18:55:21.766312 4670 leader_election.cc:288] T 230802eaaac846d59c781eb86c050d10 P 34f73195691f44ccb727078f5f420efc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 63513ded32fa459ca84ed7478e36cfbe (127.27.180.129:43763), 3a5c80e69d0f44babcc7a6e18a2b5429 (127.27.180.131:43743)
I20260811 18:55:21.769877 4365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:21.766560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.129:43763 (local address 127.27.180.130:57247)
0811 18:55:21.766679 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:21.766693 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:21.766798 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:21.767058 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:21.767062 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:21.767086 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:55:21.767209 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:21.767215 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:21.767982 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:21.767985 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:21.768788 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:21.768798 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:21.769474 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:21.769477 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:21.769589 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:21.769608 (+ 19us) client_negotiation.cc:769] Sending connection context
0811 18:55:21.769674 (+ 66us) client_negotiation.cc:240] Negotiation successful
0811 18:55:21.769747 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260811 18:55:21.769892 4655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:21.766679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.130:57247 (local address 127.27.180.129:43763)
0811 18:55:21.766820 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:21.766825 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:21.766843 (+ 18us) server_negotiation.cc:407] Connection header received
0811 18:55:21.766896 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:21.766899 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:21.766945 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:21.767019 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:55:21.767363 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:21.767874 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:21.768918 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:21.769365 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:21.769474 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:21.769532 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:21.769703 (+ 171us) server_negotiation.cc:299] Negotiation successful
0811 18:55:21.769791 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 18:55:21.770295 4660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:21.767041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.130:40381 (local address 127.27.180.131:43743)
0811 18:55:21.767170 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:21.767174 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:21.767384 (+ 210us) server_negotiation.cc:407] Connection header received
0811 18:55:21.767485 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:21.767488 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:21.767528 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:21.767588 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:55:21.767843 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:21.768318 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:21.769381 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:21.769786 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:21.769889 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:21.769939 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:21.770121 (+ 182us) server_negotiation.cc:299] Negotiation successful
0811 18:55:21.770186 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 18:55:21.770311 4677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:21.766872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.131:43743 (local address 127.27.180.130:40381)
0811 18:55:21.767292 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:21.767303 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:21.767399 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:21.767610 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:21.767612 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:21.767629 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:55:21.767741 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:21.767746 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:21.768424 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:21.768428 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:21.769262 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:21.769271 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:21.769908 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:21.769912 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:21.770009 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:21.770033 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 18:55:21.770105 (+ 72us) client_negotiation.cc:240] Negotiation successful
0811 18:55:21.770167 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":358,"mutex_wait_us":1,"thread_start_us":117,"threads_started":1}
I20260811 18:55:21.770539 4018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "230802eaaac846d59c781eb86c050d10" candidate_uuid: "34f73195691f44ccb727078f5f420efc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63513ded32fa459ca84ed7478e36cfbe" is_pre_election: true
I20260811 18:55:21.770844 4018 raft_consensus.cc:2454] T 230802eaaac846d59c781eb86c050d10 P 63513ded32fa459ca84ed7478e36cfbe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f73195691f44ccb727078f5f420efc in term 0.
I20260811 18:55:21.770910 4518 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "230802eaaac846d59c781eb86c050d10" candidate_uuid: "34f73195691f44ccb727078f5f420efc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5c80e69d0f44babcc7a6e18a2b5429" is_pre_election: true
I20260811 18:55:21.771137 4518 raft_consensus.cc:2454] T 230802eaaac846d59c781eb86c050d10 P 3a5c80e69d0f44babcc7a6e18a2b5429 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34f73195691f44ccb727078f5f420efc in term 0.
I20260811 18:55:21.771320 4147 leader_election.cc:302] T 230802eaaac846d59c781eb86c050d10 P 34f73195691f44ccb727078f5f420efc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f73195691f44ccb727078f5f420efc, 63513ded32fa459ca84ed7478e36cfbe; no voters:
I20260811 18:55:21.771591 4670 raft_consensus.cc:2781] T 230802eaaac846d59c781eb86c050d10 P 34f73195691f44ccb727078f5f420efc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:55:21.771672 4670 raft_consensus.cc:482] T 230802eaaac846d59c781eb86c050d10 P 34f73195691f44ccb727078f5f420efc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:55:21.771703 4670 raft_consensus.cc:3037] T 230802eaaac846d59c781eb86c050d10 P 34f73195691f44ccb727078f5f420efc [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:55:21.772748 4670 raft_consensus.cc:504] T 230802eaaac846d59c781eb86c050d10 P 34f73195691f44ccb727078f5f420efc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63513ded32fa459ca84ed7478e36cfbe" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 43763 } } peers { permanent_uuid: "34f73195691f44ccb727078f5f420efc" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 43603 } } peers { permanent_uuid: "3a5c80e69d0f44babcc7a6e18a2b5429" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 43743 } }
I20260811 18:55:21.773056 4670 leader_election.cc:288] T 230802eaaac846d59c781eb86c050d10 P 34f73195691f44ccb727078f5f420efc [CANDIDATE]: Term 1 election: Requested vote from peers 63513ded32fa459ca84ed7478e36cfbe (127.27.180.129:43763), 3a5c80e69d0f44babcc7a6e18a2b5429 (127.27.180.131:43743)
I20260811 18:55:21.773537 4518 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "230802eaaac846d59c781eb86c050d10" candidate_uuid: "34f73195691f44ccb727078f5f420efc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5c80e69d0f44babcc7a6e18a2b5429"
I20260811 18:55:21.773541 4018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "230802eaaac846d59c781eb86c050d10" candidate_uuid: "34f73195691f44ccb727078f5f420efc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63513ded32fa459ca84ed7478e36cfbe"
I20260811 18:55:21.773682 4018 raft_consensus.cc:3037] T 230802eaaac846d59c781eb86c050d10 P 63513ded32fa459ca84ed7478e36cfbe [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:55:21.773703 4518 raft_consensus.cc:3037] T 230802eaaac846d59c781eb86c050d10 P 3a5c80e69d0f44babcc7a6e18a2b5429 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:55:21.775049 4018 raft_consensus.cc:2454] T 230802eaaac846d59c781eb86c050d10 P 63513ded32fa459ca84ed7478e36cfbe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f73195691f44ccb727078f5f420efc in term 1.
I20260811 18:55:21.775049 4518 raft_consensus.cc:2454] T 230802eaaac846d59c781eb86c050d10 P 3a5c80e69d0f44babcc7a6e18a2b5429 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34f73195691f44ccb727078f5f420efc in term 1.
I20260811 18:55:21.775360 4147 leader_election.cc:302] T 230802eaaac846d59c781eb86c050d10 P 34f73195691f44ccb727078f5f420efc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34f73195691f44ccb727078f5f420efc, 3a5c80e69d0f44babcc7a6e18a2b5429; no voters:
I20260811 18:55:21.775532 4670 raft_consensus.cc:2781] T 230802eaaac846d59c781eb86c050d10 P 34f73195691f44ccb727078f5f420efc [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:55:21.775770 4670 raft_consensus.cc:686] T 230802eaaac846d59c781eb86c050d10 P 34f73195691f44ccb727078f5f420efc [term 1 LEADER]: Becoming Leader. State: Replica: 34f73195691f44ccb727078f5f420efc, State: Running, Role: LEADER
I20260811 18:55:21.776068 4670 consensus_queue.cc:237] T 230802eaaac846d59c781eb86c050d10 P 34f73195691f44ccb727078f5f420efc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63513ded32fa459ca84ed7478e36cfbe" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 43763 } } peers { permanent_uuid: "34f73195691f44ccb727078f5f420efc" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 43603 } } peers { permanent_uuid: "3a5c80e69d0f44babcc7a6e18a2b5429" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 43743 } }
I20260811 18:55:21.779201 3699 catalog_manager.cc:5495] T 230802eaaac846d59c781eb86c050d10 P 34f73195691f44ccb727078f5f420efc reported cstate change: term changed from 0 to 1, leader changed from <none> to 34f73195691f44ccb727078f5f420efc (127.27.180.130). New cstate: current_term: 1 leader_uuid: "34f73195691f44ccb727078f5f420efc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63513ded32fa459ca84ed7478e36cfbe" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 43763 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34f73195691f44ccb727078f5f420efc" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 43603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5c80e69d0f44babcc7a6e18a2b5429" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 43743 } health_report { overall_health: UNKNOWN } } }
I20260811 18:55:21.805015 4656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:21.801772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46588 (local address 127.27.180.130:43603)
0811 18:55:21.801891 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:21.801895 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:21.801911 (+ 16us) server_negotiation.cc:407] Connection header received
0811 18:55:21.802075 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:21.802078 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:21.802142 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:21.802228 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:55:21.802469 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:21.803003 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:21.803823 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:21.804027 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:21.804102 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:21.804353 (+ 251us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:55:21.804759 (+ 406us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:55:21.804845 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:21.804873 (+ 28us) server_negotiation.cc:299] Negotiation successful
0811 18:55:21.804920 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 18:55:21.810492 4018 raft_consensus.cc:1261] T 230802eaaac846d59c781eb86c050d10 P 63513ded32fa459ca84ed7478e36cfbe [term 1 FOLLOWER]: Refusing update from remote peer 34f73195691f44ccb727078f5f420efc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:55:21.810492 4518 raft_consensus.cc:1261] T 230802eaaac846d59c781eb86c050d10 P 3a5c80e69d0f44babcc7a6e18a2b5429 [term 1 FOLLOWER]: Refusing update from remote peer 34f73195691f44ccb727078f5f420efc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:55:21.811159 4670 consensus_queue.cc:1035] T 230802eaaac846d59c781eb86c050d10 P 34f73195691f44ccb727078f5f420efc [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5c80e69d0f44babcc7a6e18a2b5429" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 43743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:55:21.811400 4679 consensus_queue.cc:1035] T 230802eaaac846d59c781eb86c050d10 P 34f73195691f44ccb727078f5f420efc [LEADER]: Connected to new peer: Peer: permanent_uuid: "63513ded32fa459ca84ed7478e36cfbe" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 43763 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 18:55:21.816500 4631 tablet.cc:2369] T 230802eaaac846d59c781eb86c050d10 P 3a5c80e69d0f44babcc7a6e18a2b5429: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:55:21.819012 4684 mvcc.cc:205] Tried to move back new op lower bound from 7317399641328631808 to 7317399641197326336. Current Snapshot: MvccSnapshot[applied={T|T < 7317399641328631808}]
I20260811 18:55:21.820168 4687 mvcc.cc:205] Tried to move back new op lower bound from 7317399641328631808 to 7317399641197326336. Current Snapshot: MvccSnapshot[applied={T|T < 7317399641328631808}]
I20260811 18:55:21.828644 3700 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59896:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:55:21.828819 3700 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59896:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:55:21.831190 3700 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f4b773bf55ff4e398fcb775e672a743c: Sending DeleteTablet for 3 replicas of tablet 230802eaaac846d59c781eb86c050d10
I20260811 18:55:21.831957 4239 tablet_service.cc:1514] Processing DeleteTablet for tablet 230802eaaac846d59c781eb86c050d10 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:55:21 UTC) from {username='kudu', principal='kudu/127.27.180.190@KRBTEST.COM'} at 127.0.0.1:46574
I20260811 18:55:21.832165 4485 tablet_service.cc:1514] Processing DeleteTablet for tablet 230802eaaac846d59c781eb86c050d10 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:55:21 UTC) from {username='kudu', principal='kudu/127.27.180.190@KRBTEST.COM'} at 127.0.0.1:47966
I20260811 18:55:21.832489 4701 tablet_replica.cc:330] T 230802eaaac846d59c781eb86c050d10 P 34f73195691f44ccb727078f5f420efc: stopping tablet replica
I20260811 18:55:21.832445 3976 tablet_service.cc:1514] Processing DeleteTablet for tablet 230802eaaac846d59c781eb86c050d10 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:55:21 UTC) from {username='kudu', principal='kudu/127.27.180.190@KRBTEST.COM'} at 127.0.0.1:55774
I20260811 18:55:21.832852 4701 raft_consensus.cc:2229] T 230802eaaac846d59c781eb86c050d10 P 34f73195691f44ccb727078f5f420efc [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:55:21.832881 4703 tablet_replica.cc:330] T 230802eaaac846d59c781eb86c050d10 P 63513ded32fa459ca84ed7478e36cfbe: stopping tablet replica
I20260811 18:55:21.832942 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 3838
I20260811 18:55:21.833173 4703 raft_consensus.cc:2229] T 230802eaaac846d59c781eb86c050d10 P 63513ded32fa459ca84ed7478e36cfbe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:55:21.838757 4701 raft_consensus.cc:2258] T 230802eaaac846d59c781eb86c050d10 P 34f73195691f44ccb727078f5f420efc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:55:21.842042 4701 ts_tablet_manager.cc:1905] T 230802eaaac846d59c781eb86c050d10 P 34f73195691f44ccb727078f5f420efc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:55:21.845364 4701 ts_tablet_manager.cc:1918] T 230802eaaac846d59c781eb86c050d10 P 34f73195691f44ccb727078f5f420efc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:55:21.845572 4701 log.cc:1196] T 230802eaaac846d59c781eb86c050d10 P 34f73195691f44ccb727078f5f420efc: Deleting WAL directory at /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474496157106-28370-0/minicluster-data/ts-1/wal/wals/230802eaaac846d59c781eb86c050d10
I20260811 18:55:21.846103 4701 ts_tablet_manager.cc:1939] T 230802eaaac846d59c781eb86c050d10 P 34f73195691f44ccb727078f5f420efc: Deleting consensus metadata
I20260811 18:55:21.848207 4702 tablet_replica.cc:330] T 230802eaaac846d59c781eb86c050d10 P 3a5c80e69d0f44babcc7a6e18a2b5429: stopping tablet replica
I20260811 18:55:21.848225 3671 catalog_manager.cc:4841] TS 34f73195691f44ccb727078f5f420efc (127.27.180.130:43603): tablet 230802eaaac846d59c781eb86c050d10 (table test-table [id=e498b6c50ff941368597e2850978d279]) successfully deleted
I20260811 18:55:21.848551 4702 raft_consensus.cc:2229] T 230802eaaac846d59c781eb86c050d10 P 3a5c80e69d0f44babcc7a6e18a2b5429 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:55:21.848860 4702 raft_consensus.cc:2258] T 230802eaaac846d59c781eb86c050d10 P 3a5c80e69d0f44babcc7a6e18a2b5429 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:55:21.851009 4702 ts_tablet_manager.cc:1905] T 230802eaaac846d59c781eb86c050d10 P 3a5c80e69d0f44babcc7a6e18a2b5429: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:55:21.853920 4702 ts_tablet_manager.cc:1918] T 230802eaaac846d59c781eb86c050d10 P 3a5c80e69d0f44babcc7a6e18a2b5429: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
W20260811 18:55:21.853974 3674 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.27.180.129:43763 (error 108)
I20260811 18:55:21.854035 4702 log.cc:1196] T 230802eaaac846d59c781eb86c050d10 P 3a5c80e69d0f44babcc7a6e18a2b5429: Deleting WAL directory at /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786474496157106-28370-0/minicluster-data/ts-2/wal/wals/230802eaaac846d59c781eb86c050d10
I20260811 18:55:21.854403 4702 ts_tablet_manager.cc:1939] T 230802eaaac846d59c781eb86c050d10 P 3a5c80e69d0f44babcc7a6e18a2b5429: Deleting consensus metadata
I20260811 18:55:21.855158 4653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:21.854846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.129:43763 (local address 127.0.0.1:55778)
0811 18:55:21.854978 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:21.855061 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.129:43763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260811 18:55:21.855228 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 4114
W20260811 18:55:21.861516 3674 catalog_manager.cc:4584] TS 63513ded32fa459ca84ed7478e36cfbe (127.27.180.129:43763): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 230802eaaac846d59c781eb86c050d10: Network error: Client connection negotiation failed: client connection to 127.27.180.129:43763: connect: Connection refused (error 111)
I20260811 18:55:21.861776 3674 catalog_manager.cc:4841] TS 3a5c80e69d0f44babcc7a6e18a2b5429 (127.27.180.131:43743): tablet 230802eaaac846d59c781eb86c050d10 (table test-table [id=e498b6c50ff941368597e2850978d279]) successfully deleted
I20260811 18:55:21.866573 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 4368
I20260811 18:55:21.877619 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 3626
2026-08-11T18:55:21Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2500 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-11T18:55:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:55:21Z Disabled control of system clock
I20260811 18:55:21.914719 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474496157106-28370-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474496157106-28370-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474496157106-28370-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.190:44449
--webserver_interface=127.27.180.190
--webserver_port=0
--builtin_ntp_servers=127.27.180.148:39905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.180.190:44449
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 18:55:22.023095 4711 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:55:22.023344 4711 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:55:22.023378 4711 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:55:22.027302 4711 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:55:22.027356 4711 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:55:22.027371 4711 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:55:22.027386 4711 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:55:22.027400 4711 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:55:22.032374 4711 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:39905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474496157106-28370-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474496157106-28370-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.180.190:44449
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.190:44449
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.180.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4711
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474496157106-28370-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:55:22.033427 4711 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:55:22.034710 4711 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:55:22.042634 4720 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:22.042667 4728 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:22.042701 4731 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:55:22.042877 4711 server_base.cc:1034] running on GCE node
I20260811 18:55:22.043442 4711 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:55:22.044432 4711 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:55:22.045583 4711 hybrid_clock.cc:648] HybridClock initialized: now 1786474522045570 us; error 30 us; skew 500 ppm
I20260811 18:55:22.047290 4711 webserver.cc:460] Webserver started at http://127.27.180.190:33139/ using document root <none> and password file <none>
I20260811 18:55:22.047806 4711 fs_manager.cc:359] Metadata directory not provided
I20260811 18:55:22.047873 4711 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:55:22.048049 4711 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:55:22.049980 4711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474496157106-28370-0/minicluster-data/master-0/data/instance:
uuid: "54e817684d0b49d9b11f80fafbdfb817"
format_stamp: "Formatted at 2026-08-11 18:55:22 on dist-test-slave-2j7r"
server_key: "bdaad004efd775cedf6a1fb4aead3172"
server_key_iv: "59667fd564aaeb1abdeaffd6235c069b"
server_key_version: "encryptionkey@0"
I20260811 18:55:22.050496 4711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474496157106-28370-0/minicluster-data/master-0/wal/instance:
uuid: "54e817684d0b49d9b11f80fafbdfb817"
format_stamp: "Formatted at 2026-08-11 18:55:22 on dist-test-slave-2j7r"
server_key: "bdaad004efd775cedf6a1fb4aead3172"
server_key_iv: "59667fd564aaeb1abdeaffd6235c069b"
server_key_version: "encryptionkey@0"
I20260811 18:55:22.053952 4711 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:55:22.056226 4740 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:22.057382 4711 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 18:55:22.057492 4711 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474496157106-28370-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474496157106-28370-0/minicluster-data/master-0/wal
uuid: "54e817684d0b49d9b11f80fafbdfb817"
format_stamp: "Formatted at 2026-08-11 18:55:22 on dist-test-slave-2j7r"
server_key: "bdaad004efd775cedf6a1fb4aead3172"
server_key_iv: "59667fd564aaeb1abdeaffd6235c069b"
server_key_version: "encryptionkey@0"
I20260811 18:55:22.057583 4711 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474496157106-28370-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474496157106-28370-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474496157106-28370-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:55:22.062042 4711 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:55:22.062604 4711 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:55:22.062767 4711 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:55:22.069450 4711 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.190:44449
I20260811 18:55:22.069489 4828 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.190:44449 every 8 connection(s)
I20260811 18:55:22.070333 4711 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
I20260811 18:55:22.073483 4829 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:55:22.078764 4829 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54e817684d0b49d9b11f80fafbdfb817: Bootstrap starting.
I20260811 18:55:22.079969 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 4711
I20260811 18:55:22.080082 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474496157106-28370-0/minicluster-data/master-0/wal/instance
I20260811 18:55:22.080389 28370 external_mini_cluster.cc:1428] Setting key 9780fa2ec5fd5fe4f540359e84871b58
I20260811 18:55:22.081205 4829 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 54e817684d0b49d9b11f80fafbdfb817: Neither blocks nor log segments found. Creating new log.
I20260811 18:55:22.082074 4829 log.cc:824] T 00000000000000000000000000000000 P 54e817684d0b49d9b11f80fafbdfb817: Log is configured to *not* fsync() on all Append() calls
I20260811 18:55:22.084293 4829 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54e817684d0b49d9b11f80fafbdfb817: No bootstrap required, opened a new log
I20260811 18:55:22.085907 4834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:22.081560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44750 (local address 127.27.180.190:44449)
0811 18:55:22.082249 (+ 689us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:22.082269 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:22.082312 (+ 43us) server_negotiation.cc:407] Connection header received
0811 18:55:22.083015 (+ 703us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:22.083038 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:22.083380 (+ 342us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:22.083717 (+ 337us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:22.084375 (+ 658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:22.084380 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:22.084393 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:55:22.084405 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:22.084466 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:55:22.084489 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:22.084594 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:22.084711 (+ 117us) server_negotiation.cc:299] Negotiation successful
0811 18:55:22.084964 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":145,"threads_started":1}
I20260811 18:55:22.088544 4829 raft_consensus.cc:348] T 00000000000000000000000000000000 P 54e817684d0b49d9b11f80fafbdfb817 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54e817684d0b49d9b11f80fafbdfb817" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 44449 } }
I20260811 18:55:22.088745 4829 raft_consensus.cc:374] T 00000000000000000000000000000000 P 54e817684d0b49d9b11f80fafbdfb817 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:55:22.088786 4829 raft_consensus.cc:729] T 00000000000000000000000000000000 P 54e817684d0b49d9b11f80fafbdfb817 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54e817684d0b49d9b11f80fafbdfb817, State: Initialized, Role: FOLLOWER
I20260811 18:55:22.089149 4829 consensus_queue.cc:260] T 00000000000000000000000000000000 P 54e817684d0b49d9b11f80fafbdfb817 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54e817684d0b49d9b11f80fafbdfb817" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 44449 } }
I20260811 18:55:22.089280 4829 raft_consensus.cc:388] T 00000000000000000000000000000000 P 54e817684d0b49d9b11f80fafbdfb817 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:55:22.089344 4829 raft_consensus.cc:482] T 00000000000000000000000000000000 P 54e817684d0b49d9b11f80fafbdfb817 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:55:22.089432 4829 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 54e817684d0b49d9b11f80fafbdfb817 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:55:22.090314 4829 raft_consensus.cc:504] T 00000000000000000000000000000000 P 54e817684d0b49d9b11f80fafbdfb817 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54e817684d0b49d9b11f80fafbdfb817" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 44449 } }
I20260811 18:55:22.090616 4829 leader_election.cc:302] T 00000000000000000000000000000000 P 54e817684d0b49d9b11f80fafbdfb817 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 54e817684d0b49d9b11f80fafbdfb817; no voters:
I20260811 18:55:22.090873 4829 leader_election.cc:288] T 00000000000000000000000000000000 P 54e817684d0b49d9b11f80fafbdfb817 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:55:22.090981 4837 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 54e817684d0b49d9b11f80fafbdfb817 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:55:22.091217 4837 raft_consensus.cc:686] T 00000000000000000000000000000000 P 54e817684d0b49d9b11f80fafbdfb817 [term 1 LEADER]: Becoming Leader. State: Replica: 54e817684d0b49d9b11f80fafbdfb817, State: Running, Role: LEADER
I20260811 18:55:22.091487 4837 consensus_queue.cc:237] T 00000000000000000000000000000000 P 54e817684d0b49d9b11f80fafbdfb817 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54e817684d0b49d9b11f80fafbdfb817" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 44449 } }
I20260811 18:55:22.091954 4829 sys_catalog.cc:564] T 00000000000000000000000000000000 P 54e817684d0b49d9b11f80fafbdfb817 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:55:22.093066 4840 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54e817684d0b49d9b11f80fafbdfb817 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 54e817684d0b49d9b11f80fafbdfb817. Latest consensus state: current_term: 1 leader_uuid: "54e817684d0b49d9b11f80fafbdfb817" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54e817684d0b49d9b11f80fafbdfb817" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 44449 } } }
I20260811 18:55:22.093221 4840 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54e817684d0b49d9b11f80fafbdfb817 [sys.catalog]: This master's current role is: LEADER
I20260811 18:55:22.093511 4839 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54e817684d0b49d9b11f80fafbdfb817 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "54e817684d0b49d9b11f80fafbdfb817" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54e817684d0b49d9b11f80fafbdfb817" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 44449 } } }
I20260811 18:55:22.093600 4839 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54e817684d0b49d9b11f80fafbdfb817 [sys.catalog]: This master's current role is: LEADER
I20260811 18:55:22.093724 4855 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:55:22.096254 4855 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:55:22.101203 4855 catalog_manager.cc:1347] Generated new cluster ID: 4d65ea92ef4f4695ac0ac5872d956c7d
I20260811 18:55:22.101262 4855 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:55:22.123947 4855 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:55:22.124239 4855 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:55:22.133392 4855 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 54e817684d0b49d9b11f80fafbdfb817: Generated new TSK 0
I20260811 18:55:22.134001 4855 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:55:22.145285 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474496157106-28370-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474496157106-28370-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474496157106-28370-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474496157106-28370-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.180.129:0
--local_ip_for_outbound_sockets=127.27.180.129
--webserver_interface=127.27.180.129
--webserver_port=0
--tserver_master_addrs=127.27.180.190:44449
--builtin_ntp_servers=127.27.180.148:39905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 18:55:22.255182 4874 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:55:22.255414 4874 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:55:22.255452 4874 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:55:22.259358 4874 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:55:22.259404 4874 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:55:22.259456 4874 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.180.129
I20260811 18:55:22.264118 4874 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:39905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474496157106-28370-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474496157106-28370-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.180.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474496157106-28370-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.180.129
--webserver_port=0
--tserver_master_addrs=127.27.180.190:44449
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4874
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.180.129
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474496157106-28370-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:55:22.265071 4874 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:55:22.266444 4874 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:55:22.275398 4891 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:22.275465 4888 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:55:22.275480 4874 server_base.cc:1034] running on GCE node
W20260811 18:55:22.275414 4887 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:55:22.276002 4874 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:55:22.276571 4874 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:55:22.277716 4874 hybrid_clock.cc:648] HybridClock initialized: now 1786474522277699 us; error 29 us; skew 500 ppm
I20260811 18:55:22.279383 4874 webserver.cc:460] Webserver started at http://127.27.180.129:35329/ using document root <none> and password file <none>
I20260811 18:55:22.279902 4874 fs_manager.cc:359] Metadata directory not provided
I20260811 18:55:22.279966 4874 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:55:22.280145 4874 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:55:22.282074 4874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474496157106-28370-0/minicluster-data/ts-0/data/instance:
uuid: "d33795960a194c7e8e139f213b23bb49"
format_stamp: "Formatted at 2026-08-11 18:55:22 on dist-test-slave-2j7r"
server_key: "ffbeba699e91f67c97972c7036e432dd"
server_key_iv: "26b5b68d422e7953f40118d305005dd3"
server_key_version: "encryptionkey@0"
I20260811 18:55:22.282662 4874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474496157106-28370-0/minicluster-data/ts-0/wal/instance:
uuid: "d33795960a194c7e8e139f213b23bb49"
format_stamp: "Formatted at 2026-08-11 18:55:22 on dist-test-slave-2j7r"
server_key: "ffbeba699e91f67c97972c7036e432dd"
server_key_iv: "26b5b68d422e7953f40118d305005dd3"
server_key_version: "encryptionkey@0"
I20260811 18:55:22.286054 4874 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.001s
I20260811 18:55:22.288290 4903 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:22.289404 4874 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260811 18:55:22.289501 4874 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474496157106-28370-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474496157106-28370-0/minicluster-data/ts-0/wal
uuid: "d33795960a194c7e8e139f213b23bb49"
format_stamp: "Formatted at 2026-08-11 18:55:22 on dist-test-slave-2j7r"
server_key: "ffbeba699e91f67c97972c7036e432dd"
server_key_iv: "26b5b68d422e7953f40118d305005dd3"
server_key_version: "encryptionkey@0"
I20260811 18:55:22.289594 4874 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474496157106-28370-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474496157106-28370-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474496157106-28370-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:55:22.293978 4874 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:55:22.294546 4874 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:55:22.294689 4874 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:55:22.295184 4874 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:55:22.296027 4874 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:55:22.296072 4874 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:22.296111 4874 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:55:22.296139 4874 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:22.304363 4874 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.129:33089
I20260811 18:55:22.304396 5152 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.129:33089 every 8 connection(s)
I20260811 18:55:22.305241 4874 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474496157106-28370-0/minicluster-data/ts-0/data/info.pb
I20260811 18:55:22.311093 4834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:22.306930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.129:46651 (local address 127.27.180.190:44449)
0811 18:55:22.307058 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:22.307061 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:22.308061 (+ 1000us) server_negotiation.cc:407] Connection header received
0811 18:55:22.309544 (+ 1483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:22.309547 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:22.309596 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:22.309669 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:22.310665 (+ 996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:22.310668 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:22.310670 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:55:22.310683 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:22.310703 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:55:22.310712 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:22.310798 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:22.310913 (+ 115us) server_negotiation.cc:299] Negotiation successful
0811 18:55:22.310957 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 18:55:22.311708 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 4874
I20260811 18:55:22.311834 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474496157106-28370-0/minicluster-data/ts-0/wal/instance
I20260811 18:55:22.312076 5157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:22.307277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:44449 (local address 127.27.180.129:46651)
0811 18:55:22.307919 (+ 642us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:22.307959 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:22.309334 (+ 1375us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:22.309836 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:22.309843 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:22.309929 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:22.310416 (+ 487us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:55:22.310417 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:22.310456 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:55:22.310522 (+ 66us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:55:22.310782 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:55:22.310791 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:55:22.310796 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 18:55:22.310950 (+ 154us) client_negotiation.cc:240] Negotiation successful
0811 18:55:22.311182 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":116,"threads_started":1}
I20260811 18:55:22.312166 28370 external_mini_cluster.cc:1428] Setting key d5949043b4bbdc56bdbd065a1cce18f7
I20260811 18:55:22.313248 5153 heartbeater.cc:347] Connected to a master server at 127.27.180.190:44449
I20260811 18:55:22.313474 5153 heartbeater.cc:464] Registering TS with master...
I20260811 18:55:22.313908 5153 heartbeater.cc:511] Master 127.27.180.190:44449 requested a full tablet report, sending...
I20260811 18:55:22.314069 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474496157106-28370-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474496157106-28370-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474496157106-28370-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474496157106-28370-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.180.130:0
--local_ip_for_outbound_sockets=127.27.180.130
--webserver_interface=127.27.180.130
--webserver_port=0
--tserver_master_addrs=127.27.180.190:44449
--builtin_ntp_servers=127.27.180.148:39905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260811 18:55:22.315506 4772 ts_manager.cc:194] Registered new tserver with Master: d33795960a194c7e8e139f213b23bb49 (127.27.180.129:33089)
W20260811 18:55:22.424238 5159 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:55:22.424541 5159 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:55:22.424598 5159 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:55:22.428551 5159 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:55:22.428601 5159 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:55:22.428670 5159 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.180.130
I20260811 18:55:22.433717 5159 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:39905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474496157106-28370-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474496157106-28370-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.180.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474496157106-28370-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.180.130
--webserver_port=0
--tserver_master_addrs=127.27.180.190:44449
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5159
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.180.130
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474496157106-28370-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:55:22.434768 5159 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:55:22.436116 5159 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:55:22.445561 5178 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:22.445577 5167 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:22.445571 5166 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:55:22.445778 5159 server_base.cc:1034] running on GCE node
I20260811 18:55:22.446252 5159 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:55:22.446846 5159 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:55:22.448027 5159 hybrid_clock.cc:648] HybridClock initialized: now 1786474522447997 us; error 46 us; skew 500 ppm
I20260811 18:55:22.449877 5159 webserver.cc:460] Webserver started at http://127.27.180.130:37173/ using document root <none> and password file <none>
I20260811 18:55:22.450448 5159 fs_manager.cc:359] Metadata directory not provided
I20260811 18:55:22.450507 5159 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:55:22.450699 5159 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:55:22.452677 5159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474496157106-28370-0/minicluster-data/ts-1/data/instance:
uuid: "53248e1ba55b4c13a2ab2affda1e4182"
format_stamp: "Formatted at 2026-08-11 18:55:22 on dist-test-slave-2j7r"
server_key: "e03b6cbc640e2b36ef92dee1db6e040e"
server_key_iv: "d5d20f9653d7c62182d84ed755bb0212"
server_key_version: "encryptionkey@0"
I20260811 18:55:22.453250 5159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474496157106-28370-0/minicluster-data/ts-1/wal/instance:
uuid: "53248e1ba55b4c13a2ab2affda1e4182"
format_stamp: "Formatted at 2026-08-11 18:55:22 on dist-test-slave-2j7r"
server_key: "e03b6cbc640e2b36ef92dee1db6e040e"
server_key_iv: "d5d20f9653d7c62182d84ed755bb0212"
server_key_version: "encryptionkey@0"
I20260811 18:55:22.456792 5159 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:55:22.459312 5185 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:22.460551 5159 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 18:55:22.460667 5159 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474496157106-28370-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474496157106-28370-0/minicluster-data/ts-1/wal
uuid: "53248e1ba55b4c13a2ab2affda1e4182"
format_stamp: "Formatted at 2026-08-11 18:55:22 on dist-test-slave-2j7r"
server_key: "e03b6cbc640e2b36ef92dee1db6e040e"
server_key_iv: "d5d20f9653d7c62182d84ed755bb0212"
server_key_version: "encryptionkey@0"
I20260811 18:55:22.460795 5159 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474496157106-28370-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474496157106-28370-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474496157106-28370-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:55:22.465309 5159 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:55:22.465912 5159 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:55:22.466086 5159 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:55:22.466614 5159 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:55:22.467504 5159 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:55:22.467549 5159 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:22.467599 5159 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:55:22.467628 5159 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:22.476095 5159 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.130:40919
I20260811 18:55:22.476121 5421 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.130:40919 every 8 connection(s)
I20260811 18:55:22.476949 5159 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474496157106-28370-0/minicluster-data/ts-1/data/info.pb
I20260811 18:55:22.481187 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 5159
I20260811 18:55:22.481372 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474496157106-28370-0/minicluster-data/ts-1/wal/instance
I20260811 18:55:22.481712 28370 external_mini_cluster.cc:1428] Setting key ca1146964e24011cc5b8f4cbf1442e24
I20260811 18:55:22.483219 4834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:22.478749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.130:34955 (local address 127.27.180.190:44449)
0811 18:55:22.478901 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:22.478906 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:22.479948 (+ 1042us) server_negotiation.cc:407] Connection header received
0811 18:55:22.481361 (+ 1413us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:22.481365 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:22.481408 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:22.481497 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:22.482766 (+ 1269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:22.482770 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:22.482772 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:55:22.482784 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:22.482806 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:55:22.482816 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:22.482900 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:22.483049 (+ 149us) server_negotiation.cc:299] Negotiation successful
0811 18:55:22.483092 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 18:55:22.483946 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474496157106-28370-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474496157106-28370-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474496157106-28370-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474496157106-28370-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.180.131:0
--local_ip_for_outbound_sockets=127.27.180.131
--webserver_interface=127.27.180.131
--webserver_port=0
--tserver_master_addrs=127.27.180.190:44449
--builtin_ntp_servers=127.27.180.148:39905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260811 18:55:22.484284 5432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:22.479123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:44449 (local address 127.27.180.130:34955)
0811 18:55:22.479813 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:22.479853 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:22.481144 (+ 1291us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:22.481702 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:22.481711 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:22.481824 (+ 113us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:22.482458 (+ 634us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:55:22.482460 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:22.482514 (+ 54us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:55:22.482596 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:55:22.482894 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:55:22.482903 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:55:22.482910 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 18:55:22.483091 (+ 181us) client_negotiation.cc:240] Negotiation successful
0811 18:55:22.483335 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":117,"threads_started":1}
I20260811 18:55:22.485350 5425 heartbeater.cc:347] Connected to a master server at 127.27.180.190:44449
I20260811 18:55:22.485591 5425 heartbeater.cc:464] Registering TS with master...
I20260811 18:55:22.486040 5425 heartbeater.cc:511] Master 127.27.180.190:44449 requested a full tablet report, sending...
I20260811 18:55:22.487119 4772 ts_manager.cc:194] Registered new tserver with Master: 53248e1ba55b4c13a2ab2affda1e4182 (127.27.180.130:40919)
W20260811 18:55:22.595425 5433 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:55:22.595657 5433 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:55:22.595695 5433 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:55:22.599628 5433 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:55:22.599680 5433 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:55:22.599732 5433 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.180.131
I20260811 18:55:22.604322 5433 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:39905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474496157106-28370-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474496157106-28370-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.180.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474496157106-28370-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.180.131
--webserver_port=0
--tserver_master_addrs=127.27.180.190:44449
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5433
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.180.131
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474496157106-28370-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:55:22.605331 5433 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:55:22.606660 5433 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 18:55:22.615832 5433 server_base.cc:1034] running on GCE node
W20260811 18:55:22.615775 5440 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:22.615780 5446 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:22.615774 5441 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:55:22.616425 5433 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:55:22.617007 5433 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:55:22.618156 5433 hybrid_clock.cc:648] HybridClock initialized: now 1786474522618141 us; error 33 us; skew 500 ppm
I20260811 18:55:22.619876 5433 webserver.cc:460] Webserver started at http://127.27.180.131:43497/ using document root <none> and password file <none>
I20260811 18:55:22.620404 5433 fs_manager.cc:359] Metadata directory not provided
I20260811 18:55:22.620464 5433 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:55:22.620647 5433 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:55:22.622635 5433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474496157106-28370-0/minicluster-data/ts-2/data/instance:
uuid: "25318ff9a0724b2ea909c743972d8584"
format_stamp: "Formatted at 2026-08-11 18:55:22 on dist-test-slave-2j7r"
server_key: "ebee0f2793b956ab78c65125889fc551"
server_key_iv: "4ff75d04b2df8f1d59072d2d8c322458"
server_key_version: "encryptionkey@0"
I20260811 18:55:22.623152 5433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474496157106-28370-0/minicluster-data/ts-2/wal/instance:
uuid: "25318ff9a0724b2ea909c743972d8584"
format_stamp: "Formatted at 2026-08-11 18:55:22 on dist-test-slave-2j7r"
server_key: "ebee0f2793b956ab78c65125889fc551"
server_key_iv: "4ff75d04b2df8f1d59072d2d8c322458"
server_key_version: "encryptionkey@0"
I20260811 18:55:22.626668 5433 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 18:55:22.628975 5455 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:22.630206 5433 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 18:55:22.630333 5433 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474496157106-28370-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474496157106-28370-0/minicluster-data/ts-2/wal
uuid: "25318ff9a0724b2ea909c743972d8584"
format_stamp: "Formatted at 2026-08-11 18:55:22 on dist-test-slave-2j7r"
server_key: "ebee0f2793b956ab78c65125889fc551"
server_key_iv: "4ff75d04b2df8f1d59072d2d8c322458"
server_key_version: "encryptionkey@0"
I20260811 18:55:22.630424 5433 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474496157106-28370-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474496157106-28370-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474496157106-28370-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:55:22.634673 5433 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:55:22.635218 5433 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:55:22.635372 5433 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:55:22.635859 5433 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:55:22.636703 5433 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:55:22.636749 5433 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:22.636797 5433 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:55:22.636826 5433 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:22.645188 5433 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.131:33775
I20260811 18:55:22.645251 5695 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.131:33775 every 8 connection(s)
I20260811 18:55:22.646020 5433 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474496157106-28370-0/minicluster-data/ts-2/data/info.pb
I20260811 18:55:22.651196 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 5433
I20260811 18:55:22.651324 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474496157106-28370-0/minicluster-data/ts-2/wal/instance
I20260811 18:55:22.651580 28370 external_mini_cluster.cc:1428] Setting key c1c4250db9937c8152ec7b0fa2b5ef7b
I20260811 18:55:22.652045 4834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:22.647769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.131:40019 (local address 127.27.180.190:44449)
0811 18:55:22.647942 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:22.647947 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:22.648914 (+ 967us) server_negotiation.cc:407] Connection header received
0811 18:55:22.650256 (+ 1342us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:22.650259 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:22.650302 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:22.650384 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:22.651544 (+ 1160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:22.651547 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:22.651549 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:55:22.651563 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:22.651584 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:55:22.651593 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:22.651683 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:22.651851 (+ 168us) server_negotiation.cc:299] Negotiation successful
0811 18:55:22.651903 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 18:55:22.652954 5698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:22.648118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:44449 (local address 127.27.180.131:40019)
0811 18:55:22.648776 (+ 658us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:22.648814 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:22.650025 (+ 1211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:22.650529 (+ 504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:22.650536 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:22.650618 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:22.651163 (+ 545us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:55:22.651165 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:22.651210 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:55:22.651300 (+ 90us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:55:22.651686 (+ 386us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:55:22.651697 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:55:22.651703 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 18:55:22.651899 (+ 196us) client_negotiation.cc:240] Negotiation successful
0811 18:55:22.652104 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":107,"threads_started":1}
I20260811 18:55:22.653970 5696 heartbeater.cc:347] Connected to a master server at 127.27.180.190:44449
I20260811 18:55:22.654148 5696 heartbeater.cc:464] Registering TS with master...
I20260811 18:55:22.654502 5696 heartbeater.cc:511] Master 127.27.180.190:44449 requested a full tablet report, sending...
I20260811 18:55:22.655441 4772 ts_manager.cc:194] Registered new tserver with Master: 25318ff9a0724b2ea909c743972d8584 (127.27.180.131:33775)
I20260811 18:55:22.665377 28370 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 18:55:22.668704 4834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:22.667157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44756 (local address 127.27.180.190:44449)
0811 18:55:22.667288 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:22.667292 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:22.667519 (+ 227us) server_negotiation.cc:407] Connection header received
0811 18:55:22.667631 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:22.667635 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:22.667669 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:22.667730 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:22.668351 (+ 621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:22.668354 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:22.668357 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:55:22.668370 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:22.668386 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:55:22.668394 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:22.668462 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:22.668549 (+ 87us) server_negotiation.cc:299] Negotiation successful
0811 18:55:22.668588 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 18:55:22.671315 4772 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44756:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:55:22.674264 4772 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:55:22.685536 5722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:22.682348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.129:33089 (local address 127.0.0.1:33714)
0811 18:55:22.682850 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:22.682897 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:22.682993 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:22.683722 (+ 729us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:22.683728 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:22.683803 (+ 75us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:22.684766 (+ 963us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:55:22.684768 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:22.684787 (+ 19us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:55:22.684849 (+ 62us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:55:22.685240 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:55:22.685250 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:55:22.685256 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 18:55:22.685345 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 18:55:22.685411 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":63,"threads_started":1}
I20260811 18:55:22.686374 5724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:22.682517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33714 (local address 127.27.180.129:33089)
0811 18:55:22.683263 (+ 746us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:22.683276 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:22.683301 (+ 25us) server_negotiation.cc:407] Connection header received
0811 18:55:22.683356 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:22.683361 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:22.683556 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:22.684066 (+ 510us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:22.684993 (+ 927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:22.684999 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:22.685011 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:55:22.685023 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:22.685065 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:55:22.685129 (+ 64us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:22.686091 (+ 962us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:22.686142 (+ 51us) server_negotiation.cc:299] Negotiation successful
0811 18:55:22.686227 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":626,"thread_start_us":106,"threads_started":1}
I20260811 18:55:22.687044 5718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:22.682517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.130:40919 (local address 127.0.0.1:56530)
0811 18:55:22.682949 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:22.682961 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:22.683015 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:22.685598 (+ 2583us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:22.685601 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:22.685617 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:22.686099 (+ 482us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:55:22.686101 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:22.686113 (+ 12us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:55:22.686143 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:55:22.686572 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:55:22.686575 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:55:22.686579 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 18:55:22.686657 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 18:55:22.686697 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":109,"threads_started":1}
I20260811 18:55:22.687601 5732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:22.684207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56530 (local address 127.27.180.130:40919)
0811 18:55:22.684578 (+ 371us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:22.684590 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:22.684616 (+ 26us) server_negotiation.cc:407] Connection header received
0811 18:55:22.684669 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:22.684674 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:22.684836 (+ 162us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:22.684964 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:22.686304 (+ 1340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:22.686310 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:22.686322 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:55:22.686338 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:22.686383 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:55:22.686420 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:22.686523 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:22.687364 (+ 841us) server_negotiation.cc:299] Negotiation successful
0811 18:55:22.687450 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":256,"thread_start_us":126,"threads_started":1}
I20260811 18:55:22.689378 5009 tablet_service.cc:1467] Processing CreateTablet for tablet 79b18089c0cd4bf2a35718cb03f143a1 (DEFAULT_TABLE table=test-table [id=7348a85f56874e708bd0e275ae2ca262]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:55:22.690501 5730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:22.683018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.131:33775 (local address 127.0.0.1:36096)
0811 18:55:22.683742 (+ 724us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:22.683757 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:22.683850 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:22.685831 (+ 1981us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:22.685834 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:22.685848 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:22.689871 (+ 4023us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:55:22.689873 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:22.689883 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:55:22.689908 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:55:22.690283 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:55:22.690286 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:55:22.690289 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 18:55:22.690355 (+ 66us) client_negotiation.cc:240] Negotiation successful
0811 18:55:22.690388 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":659,"thread_start_us":87,"threads_started":1}
I20260811 18:55:22.690528 5009 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79b18089c0cd4bf2a35718cb03f143a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:55:22.690856 5731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:22.683323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36096 (local address 127.27.180.131:33775)
0811 18:55:22.684660 (+ 1337us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:22.684671 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:22.684693 (+ 22us) server_negotiation.cc:407] Connection header received
0811 18:55:22.684759 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:22.684764 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:22.684931 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:22.685050 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:22.690042 (+ 4992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:22.690048 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:22.690059 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:55:22.690071 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:22.690113 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:55:22.690148 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:22.690241 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:22.690640 (+ 399us) server_negotiation.cc:299] Negotiation successful
0811 18:55:22.690718 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1236,"thread_start_us":154,"threads_started":1}
I20260811 18:55:22.693809 5541 tablet_service.cc:1467] Processing CreateTablet for tablet 79b18089c0cd4bf2a35718cb03f143a1 (DEFAULT_TABLE table=test-table [id=7348a85f56874e708bd0e275ae2ca262]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:55:22.694844 5541 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79b18089c0cd4bf2a35718cb03f143a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:55:22.694744 5303 tablet_service.cc:1467] Processing CreateTablet for tablet 79b18089c0cd4bf2a35718cb03f143a1 (DEFAULT_TABLE table=test-table [id=7348a85f56874e708bd0e275ae2ca262]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:55:22.695996 5303 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79b18089c0cd4bf2a35718cb03f143a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:55:22.698621 5733 tablet_bootstrap.cc:492] T 79b18089c0cd4bf2a35718cb03f143a1 P d33795960a194c7e8e139f213b23bb49: Bootstrap starting.
I20260811 18:55:22.700964 5733 tablet_bootstrap.cc:654] T 79b18089c0cd4bf2a35718cb03f143a1 P d33795960a194c7e8e139f213b23bb49: Neither blocks nor log segments found. Creating new log.
I20260811 18:55:22.701882 5733 log.cc:824] T 79b18089c0cd4bf2a35718cb03f143a1 P d33795960a194c7e8e139f213b23bb49: Log is configured to *not* fsync() on all Append() calls
I20260811 18:55:22.704478 5742 tablet_bootstrap.cc:492] T 79b18089c0cd4bf2a35718cb03f143a1 P 53248e1ba55b4c13a2ab2affda1e4182: Bootstrap starting.
I20260811 18:55:22.704708 5734 tablet_bootstrap.cc:492] T 79b18089c0cd4bf2a35718cb03f143a1 P 25318ff9a0724b2ea909c743972d8584: Bootstrap starting.
I20260811 18:55:22.706943 5733 tablet_bootstrap.cc:492] T 79b18089c0cd4bf2a35718cb03f143a1 P d33795960a194c7e8e139f213b23bb49: No bootstrap required, opened a new log
I20260811 18:55:22.707059 5734 tablet_bootstrap.cc:654] T 79b18089c0cd4bf2a35718cb03f143a1 P 25318ff9a0724b2ea909c743972d8584: Neither blocks nor log segments found. Creating new log.
I20260811 18:55:22.707162 5733 ts_tablet_manager.cc:1397] T 79b18089c0cd4bf2a35718cb03f143a1 P d33795960a194c7e8e139f213b23bb49: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.004s
I20260811 18:55:22.707527 5742 tablet_bootstrap.cc:654] T 79b18089c0cd4bf2a35718cb03f143a1 P 53248e1ba55b4c13a2ab2affda1e4182: Neither blocks nor log segments found. Creating new log.
I20260811 18:55:22.708037 5734 log.cc:824] T 79b18089c0cd4bf2a35718cb03f143a1 P 25318ff9a0724b2ea909c743972d8584: Log is configured to *not* fsync() on all Append() calls
I20260811 18:55:22.708321 5742 log.cc:824] T 79b18089c0cd4bf2a35718cb03f143a1 P 53248e1ba55b4c13a2ab2affda1e4182: Log is configured to *not* fsync() on all Append() calls
I20260811 18:55:22.710095 5734 tablet_bootstrap.cc:492] T 79b18089c0cd4bf2a35718cb03f143a1 P 25318ff9a0724b2ea909c743972d8584: No bootstrap required, opened a new log
I20260811 18:55:22.710095 5742 tablet_bootstrap.cc:492] T 79b18089c0cd4bf2a35718cb03f143a1 P 53248e1ba55b4c13a2ab2affda1e4182: No bootstrap required, opened a new log
I20260811 18:55:22.710279 5734 ts_tablet_manager.cc:1397] T 79b18089c0cd4bf2a35718cb03f143a1 P 25318ff9a0724b2ea909c743972d8584: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 18:55:22.710278 5742 ts_tablet_manager.cc:1397] T 79b18089c0cd4bf2a35718cb03f143a1 P 53248e1ba55b4c13a2ab2affda1e4182: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 18:55:22.711604 5733 raft_consensus.cc:348] T 79b18089c0cd4bf2a35718cb03f143a1 P d33795960a194c7e8e139f213b23bb49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d33795960a194c7e8e139f213b23bb49" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 33089 } } peers { permanent_uuid: "53248e1ba55b4c13a2ab2affda1e4182" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 40919 } } peers { permanent_uuid: "25318ff9a0724b2ea909c743972d8584" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 33775 } }
I20260811 18:55:22.711840 5733 raft_consensus.cc:374] T 79b18089c0cd4bf2a35718cb03f143a1 P d33795960a194c7e8e139f213b23bb49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:55:22.711887 5733 raft_consensus.cc:729] T 79b18089c0cd4bf2a35718cb03f143a1 P d33795960a194c7e8e139f213b23bb49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d33795960a194c7e8e139f213b23bb49, State: Initialized, Role: FOLLOWER
I20260811 18:55:22.712385 5733 consensus_queue.cc:260] T 79b18089c0cd4bf2a35718cb03f143a1 P d33795960a194c7e8e139f213b23bb49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d33795960a194c7e8e139f213b23bb49" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 33089 } } peers { permanent_uuid: "53248e1ba55b4c13a2ab2affda1e4182" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 40919 } } peers { permanent_uuid: "25318ff9a0724b2ea909c743972d8584" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 33775 } }
I20260811 18:55:22.713243 5153 heartbeater.cc:503] Master 127.27.180.190:44449 was elected leader, sending a full tablet report...
I20260811 18:55:22.713428 5733 ts_tablet_manager.cc:1428] T 79b18089c0cd4bf2a35718cb03f143a1 P d33795960a194c7e8e139f213b23bb49: Time spent starting tablet: real 0.006s user 0.006s sys 0.002s
I20260811 18:55:22.714695 5742 raft_consensus.cc:348] T 79b18089c0cd4bf2a35718cb03f143a1 P 53248e1ba55b4c13a2ab2affda1e4182 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d33795960a194c7e8e139f213b23bb49" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 33089 } } peers { permanent_uuid: "53248e1ba55b4c13a2ab2affda1e4182" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 40919 } } peers { permanent_uuid: "25318ff9a0724b2ea909c743972d8584" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 33775 } }
I20260811 18:55:22.714695 5734 raft_consensus.cc:348] T 79b18089c0cd4bf2a35718cb03f143a1 P 25318ff9a0724b2ea909c743972d8584 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d33795960a194c7e8e139f213b23bb49" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 33089 } } peers { permanent_uuid: "53248e1ba55b4c13a2ab2affda1e4182" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 40919 } } peers { permanent_uuid: "25318ff9a0724b2ea909c743972d8584" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 33775 } }
I20260811 18:55:22.714939 5742 raft_consensus.cc:374] T 79b18089c0cd4bf2a35718cb03f143a1 P 53248e1ba55b4c13a2ab2affda1e4182 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:55:22.714939 5734 raft_consensus.cc:374] T 79b18089c0cd4bf2a35718cb03f143a1 P 25318ff9a0724b2ea909c743972d8584 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:55:22.714986 5734 raft_consensus.cc:729] T 79b18089c0cd4bf2a35718cb03f143a1 P 25318ff9a0724b2ea909c743972d8584 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25318ff9a0724b2ea909c743972d8584, State: Initialized, Role: FOLLOWER
I20260811 18:55:22.714986 5742 raft_consensus.cc:729] T 79b18089c0cd4bf2a35718cb03f143a1 P 53248e1ba55b4c13a2ab2affda1e4182 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53248e1ba55b4c13a2ab2affda1e4182, State: Initialized, Role: FOLLOWER
I20260811 18:55:22.715430 5734 consensus_queue.cc:260] T 79b18089c0cd4bf2a35718cb03f143a1 P 25318ff9a0724b2ea909c743972d8584 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d33795960a194c7e8e139f213b23bb49" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 33089 } } peers { permanent_uuid: "53248e1ba55b4c13a2ab2affda1e4182" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 40919 } } peers { permanent_uuid: "25318ff9a0724b2ea909c743972d8584" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 33775 } }
I20260811 18:55:22.715457 5742 consensus_queue.cc:260] T 79b18089c0cd4bf2a35718cb03f143a1 P 53248e1ba55b4c13a2ab2affda1e4182 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d33795960a194c7e8e139f213b23bb49" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 33089 } } peers { permanent_uuid: "53248e1ba55b4c13a2ab2affda1e4182" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 40919 } } peers { permanent_uuid: "25318ff9a0724b2ea909c743972d8584" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 33775 } }
I20260811 18:55:22.716373 5742 ts_tablet_manager.cc:1428] T 79b18089c0cd4bf2a35718cb03f143a1 P 53248e1ba55b4c13a2ab2affda1e4182: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 18:55:22.716373 5734 ts_tablet_manager.cc:1428] T 79b18089c0cd4bf2a35718cb03f143a1 P 25318ff9a0724b2ea909c743972d8584: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 18:55:22.716693 5696 heartbeater.cc:503] Master 127.27.180.190:44449 was elected leader, sending a full tablet report...
I20260811 18:55:22.717447 5425 heartbeater.cc:503] Master 127.27.180.190:44449 was elected leader, sending a full tablet report...
W20260811 18:55:22.727856 5431 tablet.cc:2369] T 79b18089c0cd4bf2a35718cb03f143a1 P 53248e1ba55b4c13a2ab2affda1e4182: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:55:22.750753 5752 raft_consensus.cc:482] T 79b18089c0cd4bf2a35718cb03f143a1 P 53248e1ba55b4c13a2ab2affda1e4182 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:55:22.750986 5752 raft_consensus.cc:504] T 79b18089c0cd4bf2a35718cb03f143a1 P 53248e1ba55b4c13a2ab2affda1e4182 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d33795960a194c7e8e139f213b23bb49" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 33089 } } peers { permanent_uuid: "53248e1ba55b4c13a2ab2affda1e4182" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 40919 } } peers { permanent_uuid: "25318ff9a0724b2ea909c743972d8584" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 33775 } }
I20260811 18:55:22.752444 5752 leader_election.cc:288] T 79b18089c0cd4bf2a35718cb03f143a1 P 53248e1ba55b4c13a2ab2affda1e4182 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d33795960a194c7e8e139f213b23bb49 (127.27.180.129:33089), 25318ff9a0724b2ea909c743972d8584 (127.27.180.131:33775)
I20260811 18:55:22.754609 5731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:22.752907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.130:43883 (local address 127.27.180.131:33775)
0811 18:55:22.753010 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:22.753014 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:22.753026 (+ 12us) server_negotiation.cc:407] Connection header received
0811 18:55:22.753067 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:22.753069 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:22.753132 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:22.753205 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:22.754225 (+ 1020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:22.754228 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:22.754230 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:55:22.754242 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:22.754268 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:55:22.754276 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:22.754362 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:22.754447 (+ 85us) server_negotiation.cc:299] Negotiation successful
0811 18:55:22.754484 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 18:55:22.754609 5432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:22.752627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.131:33775 (local address 127.27.180.130:43883)
0811 18:55:22.752768 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:22.752784 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:22.752878 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:22.753242 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:22.753245 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:22.753264 (+ 19us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:22.754056 (+ 792us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:55:22.754058 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:22.754069 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:55:22.754096 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:55:22.754361 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:55:22.754365 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:55:22.754368 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 18:55:22.754445 (+ 77us) client_negotiation.cc:240] Negotiation successful
0811 18:55:22.754483 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260811 18:55:22.755246 5591 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79b18089c0cd4bf2a35718cb03f143a1" candidate_uuid: "53248e1ba55b4c13a2ab2affda1e4182" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25318ff9a0724b2ea909c743972d8584" is_pre_election: true
I20260811 18:55:22.755483 5591 raft_consensus.cc:2454] T 79b18089c0cd4bf2a35718cb03f143a1 P 25318ff9a0724b2ea909c743972d8584 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53248e1ba55b4c13a2ab2affda1e4182 in term 0.
I20260811 18:55:22.755976 5192 leader_election.cc:302] T 79b18089c0cd4bf2a35718cb03f143a1 P 53248e1ba55b4c13a2ab2affda1e4182 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25318ff9a0724b2ea909c743972d8584, 53248e1ba55b4c13a2ab2affda1e4182; no voters:
I20260811 18:55:22.756274 5752 raft_consensus.cc:2781] T 79b18089c0cd4bf2a35718cb03f143a1 P 53248e1ba55b4c13a2ab2affda1e4182 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:55:22.756352 5752 raft_consensus.cc:482] T 79b18089c0cd4bf2a35718cb03f143a1 P 53248e1ba55b4c13a2ab2affda1e4182 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:55:22.756417 5752 raft_consensus.cc:3037] T 79b18089c0cd4bf2a35718cb03f143a1 P 53248e1ba55b4c13a2ab2affda1e4182 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:55:22.757000 5753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:22.752627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.129:33089 (local address 127.27.180.130:49755)
0811 18:55:22.753589 (+ 962us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:22.753600 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:22.753683 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:22.753904 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:22.753908 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:22.753925 (+ 17us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:22.756319 (+ 2394us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:55:22.756321 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:22.756331 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:55:22.756360 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:55:22.756713 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:55:22.756716 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:55:22.756718 (+ 2us) client_negotiation.cc:769] Sending connection context
0811 18:55:22.756807 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 18:55:22.756845 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":904,"mutex_wait_us":653,"thread_start_us":97,"threads_started":1}
I20260811 18:55:22.757000 5724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:22.752839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.130:49755 (local address 127.27.180.129:33089)
0811 18:55:22.752965 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:22.752970 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:22.753680 (+ 710us) server_negotiation.cc:407] Connection header received
0811 18:55:22.753758 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:22.753763 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:22.753802 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:22.753871 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:22.756520 (+ 2649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:22.756524 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:22.756527 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:55:22.756543 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:22.756564 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:55:22.756574 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:22.756641 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:22.756835 (+ 194us) server_negotiation.cc:299] Negotiation successful
0811 18:55:22.756882 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 18:55:22.757694 5752 raft_consensus.cc:504] T 79b18089c0cd4bf2a35718cb03f143a1 P 53248e1ba55b4c13a2ab2affda1e4182 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d33795960a194c7e8e139f213b23bb49" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 33089 } } peers { permanent_uuid: "53248e1ba55b4c13a2ab2affda1e4182" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 40919 } } peers { permanent_uuid: "25318ff9a0724b2ea909c743972d8584" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 33775 } }
I20260811 18:55:22.757890 5055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79b18089c0cd4bf2a35718cb03f143a1" candidate_uuid: "53248e1ba55b4c13a2ab2affda1e4182" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d33795960a194c7e8e139f213b23bb49" is_pre_election: true
I20260811 18:55:22.758077 5752 leader_election.cc:288] T 79b18089c0cd4bf2a35718cb03f143a1 P 53248e1ba55b4c13a2ab2affda1e4182 [CANDIDATE]: Term 1 election: Requested vote from peers d33795960a194c7e8e139f213b23bb49 (127.27.180.129:33089), 25318ff9a0724b2ea909c743972d8584 (127.27.180.131:33775)
I20260811 18:55:22.758177 5055 raft_consensus.cc:2454] T 79b18089c0cd4bf2a35718cb03f143a1 P d33795960a194c7e8e139f213b23bb49 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53248e1ba55b4c13a2ab2affda1e4182 in term 0.
I20260811 18:55:22.758307 5053 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79b18089c0cd4bf2a35718cb03f143a1" candidate_uuid: "53248e1ba55b4c13a2ab2affda1e4182" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d33795960a194c7e8e139f213b23bb49"
I20260811 18:55:22.758421 5053 raft_consensus.cc:3037] T 79b18089c0cd4bf2a35718cb03f143a1 P d33795960a194c7e8e139f213b23bb49 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:55:22.758442 5591 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79b18089c0cd4bf2a35718cb03f143a1" candidate_uuid: "53248e1ba55b4c13a2ab2affda1e4182" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25318ff9a0724b2ea909c743972d8584"
I20260811 18:55:22.758544 5591 raft_consensus.cc:3037] T 79b18089c0cd4bf2a35718cb03f143a1 P 25318ff9a0724b2ea909c743972d8584 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:55:22.759717 5591 raft_consensus.cc:2454] T 79b18089c0cd4bf2a35718cb03f143a1 P 25318ff9a0724b2ea909c743972d8584 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53248e1ba55b4c13a2ab2affda1e4182 in term 1.
I20260811 18:55:22.759716 5053 raft_consensus.cc:2454] T 79b18089c0cd4bf2a35718cb03f143a1 P d33795960a194c7e8e139f213b23bb49 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53248e1ba55b4c13a2ab2affda1e4182 in term 1.
I20260811 18:55:22.760121 5192 leader_election.cc:302] T 79b18089c0cd4bf2a35718cb03f143a1 P 53248e1ba55b4c13a2ab2affda1e4182 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 25318ff9a0724b2ea909c743972d8584, 53248e1ba55b4c13a2ab2affda1e4182; no voters:
I20260811 18:55:22.760283 5752 raft_consensus.cc:2781] T 79b18089c0cd4bf2a35718cb03f143a1 P 53248e1ba55b4c13a2ab2affda1e4182 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:55:22.760461 5752 raft_consensus.cc:686] T 79b18089c0cd4bf2a35718cb03f143a1 P 53248e1ba55b4c13a2ab2affda1e4182 [term 1 LEADER]: Becoming Leader. State: Replica: 53248e1ba55b4c13a2ab2affda1e4182, State: Running, Role: LEADER
I20260811 18:55:22.760767 5752 consensus_queue.cc:237] T 79b18089c0cd4bf2a35718cb03f143a1 P 53248e1ba55b4c13a2ab2affda1e4182 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d33795960a194c7e8e139f213b23bb49" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 33089 } } peers { permanent_uuid: "53248e1ba55b4c13a2ab2affda1e4182" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 40919 } } peers { permanent_uuid: "25318ff9a0724b2ea909c743972d8584" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 33775 } }
I20260811 18:55:22.763763 4757 catalog_manager.cc:5495] T 79b18089c0cd4bf2a35718cb03f143a1 P 53248e1ba55b4c13a2ab2affda1e4182 reported cstate change: term changed from 0 to 1, leader changed from <none> to 53248e1ba55b4c13a2ab2affda1e4182 (127.27.180.130). New cstate: current_term: 1 leader_uuid: "53248e1ba55b4c13a2ab2affda1e4182" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d33795960a194c7e8e139f213b23bb49" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 33089 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53248e1ba55b4c13a2ab2affda1e4182" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 40919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "25318ff9a0724b2ea909c743972d8584" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 33775 } health_report { overall_health: UNKNOWN } } }
I20260811 18:55:22.772262 5732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:22.771168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56544 (local address 127.27.180.130:40919)
0811 18:55:22.771266 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:22.771269 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:22.771315 (+ 46us) server_negotiation.cc:407] Connection header received
0811 18:55:22.771439 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:22.771441 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:22.771493 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:22.771559 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:22.772026 (+ 467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:22.772029 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:22.772031 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:55:22.772041 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:22.772057 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:55:22.772064 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:22.772127 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:22.772153 (+ 26us) server_negotiation.cc:299] Negotiation successful
0811 18:55:22.772181 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260811 18:55:22.777565 5053 raft_consensus.cc:1261] T 79b18089c0cd4bf2a35718cb03f143a1 P d33795960a194c7e8e139f213b23bb49 [term 1 FOLLOWER]: Refusing update from remote peer 53248e1ba55b4c13a2ab2affda1e4182: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:55:22.777565 5591 raft_consensus.cc:1261] T 79b18089c0cd4bf2a35718cb03f143a1 P 25318ff9a0724b2ea909c743972d8584 [term 1 FOLLOWER]: Refusing update from remote peer 53248e1ba55b4c13a2ab2affda1e4182: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:55:22.778259 5752 consensus_queue.cc:1035] T 79b18089c0cd4bf2a35718cb03f143a1 P 53248e1ba55b4c13a2ab2affda1e4182 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d33795960a194c7e8e139f213b23bb49" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 33089 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:55:22.778432 5754 consensus_queue.cc:1035] T 79b18089c0cd4bf2a35718cb03f143a1 P 53248e1ba55b4c13a2ab2affda1e4182 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25318ff9a0724b2ea909c743972d8584" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 33775 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:55:22.793386 4772 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44756:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:55:22.793565 4772 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44756:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:55:22.796000 4772 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 54e817684d0b49d9b11f80fafbdfb817: Sending DeleteTablet for 3 replicas of tablet 79b18089c0cd4bf2a35718cb03f143a1
I20260811 18:55:22.796804 5009 tablet_service.cc:1514] Processing DeleteTablet for tablet 79b18089c0cd4bf2a35718cb03f143a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:55:22 UTC) from {username='slave'} at 127.0.0.1:33714
I20260811 18:55:22.796831 5303 tablet_service.cc:1514] Processing DeleteTablet for tablet 79b18089c0cd4bf2a35718cb03f143a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:55:22 UTC) from {username='slave'} at 127.0.0.1:56530
I20260811 18:55:22.796823 5541 tablet_service.cc:1514] Processing DeleteTablet for tablet 79b18089c0cd4bf2a35718cb03f143a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:55:22 UTC) from {username='slave'} at 127.0.0.1:36096
I20260811 18:55:22.797272 5778 tablet_replica.cc:330] T 79b18089c0cd4bf2a35718cb03f143a1 P 53248e1ba55b4c13a2ab2affda1e4182: stopping tablet replica
I20260811 18:55:22.797487 5777 tablet_replica.cc:330] T 79b18089c0cd4bf2a35718cb03f143a1 P d33795960a194c7e8e139f213b23bb49: stopping tablet replica
I20260811 18:55:22.797559 5778 raft_consensus.cc:2229] T 79b18089c0cd4bf2a35718cb03f143a1 P 53248e1ba55b4c13a2ab2affda1e4182 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:55:22.797890 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 4874
I20260811 18:55:22.816308 5778 raft_consensus.cc:2258] T 79b18089c0cd4bf2a35718cb03f143a1 P 53248e1ba55b4c13a2ab2affda1e4182 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:55:22.825811 5778 ts_tablet_manager.cc:1905] T 79b18089c0cd4bf2a35718cb03f143a1 P 53248e1ba55b4c13a2ab2affda1e4182: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:55:22.829262 5778 ts_tablet_manager.cc:1918] T 79b18089c0cd4bf2a35718cb03f143a1 P 53248e1ba55b4c13a2ab2affda1e4182: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:55:22.829367 5775 tablet_replica.cc:330] T 79b18089c0cd4bf2a35718cb03f143a1 P 25318ff9a0724b2ea909c743972d8584: stopping tablet replica
I20260811 18:55:22.829392 5778 log.cc:1196] T 79b18089c0cd4bf2a35718cb03f143a1 P 53248e1ba55b4c13a2ab2affda1e4182: Deleting WAL directory at /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474496157106-28370-0/minicluster-data/ts-1/wal/wals/79b18089c0cd4bf2a35718cb03f143a1
I20260811 18:55:22.829733 5775 raft_consensus.cc:2229] T 79b18089c0cd4bf2a35718cb03f143a1 P 25318ff9a0724b2ea909c743972d8584 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:55:22.829806 5778 ts_tablet_manager.cc:1939] T 79b18089c0cd4bf2a35718cb03f143a1 P 53248e1ba55b4c13a2ab2affda1e4182: Deleting consensus metadata
I20260811 18:55:22.830121 5775 raft_consensus.cc:2258] T 79b18089c0cd4bf2a35718cb03f143a1 P 25318ff9a0724b2ea909c743972d8584 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:55:22.831358 4746 catalog_manager.cc:4841] TS 53248e1ba55b4c13a2ab2affda1e4182 (127.27.180.130:40919): tablet 79b18089c0cd4bf2a35718cb03f143a1 (table test-table [id=7348a85f56874e708bd0e275ae2ca262]) successfully deleted
I20260811 18:55:22.831601 5775 ts_tablet_manager.cc:1905] T 79b18089c0cd4bf2a35718cb03f143a1 P 25318ff9a0724b2ea909c743972d8584: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:55:22.834141 5775 ts_tablet_manager.cc:1918] T 79b18089c0cd4bf2a35718cb03f143a1 P 25318ff9a0724b2ea909c743972d8584: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:55:22.834250 5775 log.cc:1196] T 79b18089c0cd4bf2a35718cb03f143a1 P 25318ff9a0724b2ea909c743972d8584: Deleting WAL directory at /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786474496157106-28370-0/minicluster-data/ts-2/wal/wals/79b18089c0cd4bf2a35718cb03f143a1
I20260811 18:55:22.834563 5775 ts_tablet_manager.cc:1939] T 79b18089c0cd4bf2a35718cb03f143a1 P 25318ff9a0724b2ea909c743972d8584: Deleting consensus metadata
W20260811 18:55:22.835065 4745 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.27.180.129:33089 (error 108)
I20260811 18:55:22.835668 4746 catalog_manager.cc:4841] TS 25318ff9a0724b2ea909c743972d8584 (127.27.180.131:33775): tablet 79b18089c0cd4bf2a35718cb03f143a1 (table test-table [id=7348a85f56874e708bd0e275ae2ca262]) successfully deleted
I20260811 18:55:22.836229 5730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:22.835955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.129:33089 (local address 127.0.0.1:33728)
0811 18:55:22.836079 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:22.836149 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.129:33089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260811 18:55:22.836381 4745 catalog_manager.cc:4584] TS d33795960a194c7e8e139f213b23bb49 (127.27.180.129:33089): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 79b18089c0cd4bf2a35718cb03f143a1: Network error: Client connection negotiation failed: client connection to 127.27.180.129:33089: connect: Connection refused (error 111)
I20260811 18:55:22.836416 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 5159
I20260811 18:55:22.852340 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 5433
I20260811 18:55:22.870498 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 4711
2026-08-11T18:55:22Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (993 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260811 18:55:22.900321 28370 mini_oidc.cc:152] Starting JWKS server
I20260811 18:55:22.902247 28370 webserver.cc:460] Webserver started at https://127.0.0.1:36585/ using document root <none> and password file <none>
I20260811 18:55:22.902400 28370 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 18:55:22.902648 28370 webserver.cc:460] Webserver started at http://127.0.0.1:41797/ using document root <none> and password file <none>
2026-08-11T18:55:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:55:22Z Disabled control of system clock
I20260811 18:55:22.912187 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786474496157106-28370-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786474496157106-28370-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786474496157106-28370-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.190:42453
--webserver_interface=127.27.180.190
--webserver_port=0
--builtin_ntp_servers=127.27.180.148:36021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.180.190:42453
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36585/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786474496157106-28370-0/testchainca.cert with env {}
W20260811 18:55:23.021485 5799 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:55:23.021718 5799 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:55:23.021754 5799 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:55:23.025627 5799 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:55:23.025676 5799 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:55:23.025689 5799 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:55:23.025704 5799 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:55:23.025717 5799 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 18:55:23.025736 5799 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 18:55:23.025758 5799 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:36585/jwks.json
I20260811 18:55:23.030994 5799 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:36021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786474496157106-28370-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786474496157106-28370-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.180.190:42453
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.190:42453
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36585/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.180.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786474496157106-28370-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5799
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786474496157106-28370-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:55:23.031888 5799 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:55:23.032791 5799 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:55:23.040795 5806 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:23.040807 5810 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:23.040807 5807 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:55:23.041059 5799 server_base.cc:1034] running on GCE node
I20260811 18:55:23.041755 5799 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:55:23.042778 5799 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:55:23.043964 5799 hybrid_clock.cc:648] HybridClock initialized: now 1786474523043950 us; error 25 us; skew 500 ppm
I20260811 18:55:23.045742 5799 webserver.cc:460] Webserver started at http://127.27.180.190:33421/ using document root <none> and password file <none>
I20260811 18:55:23.046289 5799 fs_manager.cc:359] Metadata directory not provided
I20260811 18:55:23.046355 5799 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:55:23.046550 5799 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:55:23.048472 5799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786474496157106-28370-0/minicluster-data/master-0/data/instance:
uuid: "7dd86ba9692c4130af502c2d4242c641"
format_stamp: "Formatted at 2026-08-11 18:55:23 on dist-test-slave-2j7r"
server_key: "8b5f19eb2f1a8939fe59f37ded11d7f4"
server_key_iv: "245bc28ea193b0ab556eb8f3f3c48347"
server_key_version: "encryptionkey@0"
I20260811 18:55:23.049000 5799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786474496157106-28370-0/minicluster-data/master-0/wal/instance:
uuid: "7dd86ba9692c4130af502c2d4242c641"
format_stamp: "Formatted at 2026-08-11 18:55:23 on dist-test-slave-2j7r"
server_key: "8b5f19eb2f1a8939fe59f37ded11d7f4"
server_key_iv: "245bc28ea193b0ab556eb8f3f3c48347"
server_key_version: "encryptionkey@0"
I20260811 18:55:23.052434 5799 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260811 18:55:23.054764 5825 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:23.055984 5799 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260811 18:55:23.056107 5799 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786474496157106-28370-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786474496157106-28370-0/minicluster-data/master-0/wal
uuid: "7dd86ba9692c4130af502c2d4242c641"
format_stamp: "Formatted at 2026-08-11 18:55:23 on dist-test-slave-2j7r"
server_key: "8b5f19eb2f1a8939fe59f37ded11d7f4"
server_key_iv: "245bc28ea193b0ab556eb8f3f3c48347"
server_key_version: "encryptionkey@0"
I20260811 18:55:23.056200 5799 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786474496157106-28370-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786474496157106-28370-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786474496157106-28370-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:55:23.077771 5799 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:55:23.078414 5799 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:55:23.078580 5799 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:55:23.085261 5799 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.190:42453
I20260811 18:55:23.085289 5931 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.190:42453 every 8 connection(s)
I20260811 18:55:23.086146 5799 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
I20260811 18:55:23.087589 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 5799
I20260811 18:55:23.087730 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786474496157106-28370-0/minicluster-data/master-0/wal/instance
I20260811 18:55:23.088076 28370 external_mini_cluster.cc:1428] Setting key a17533c10530a313d473d957c73bfdde
I20260811 18:55:23.089705 5933 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:55:23.095252 5933 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7dd86ba9692c4130af502c2d4242c641: Bootstrap starting.
I20260811 18:55:23.096626 5941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:23.089491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40960 (local address 127.27.180.190:42453)
0811 18:55:23.090136 (+ 645us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:23.090151 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:23.090185 (+ 34us) server_negotiation.cc:407] Connection header received
0811 18:55:23.090833 (+ 648us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:23.090848 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:23.091207 (+ 359us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:23.091506 (+ 299us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:23.091904 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:23.092818 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:23.093709 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:23.094156 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:23.094308 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:23.095044 (+ 736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:23.095048 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:23.095059 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:55:23.095452 (+ 393us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:23.095506 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:55:23.095526 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:23.095608 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:23.095740 (+ 132us) server_negotiation.cc:299] Negotiation successful
0811 18:55:23.095971 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":161,"threads_started":1}
I20260811 18:55:23.097561 5933 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7dd86ba9692c4130af502c2d4242c641: Neither blocks nor log segments found. Creating new log.
I20260811 18:55:23.098337 5933 log.cc:824] T 00000000000000000000000000000000 P 7dd86ba9692c4130af502c2d4242c641: Log is configured to *not* fsync() on all Append() calls
I20260811 18:55:23.099088 28370 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:55:23.100140 5933 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7dd86ba9692c4130af502c2d4242c641: No bootstrap required, opened a new log
I20260811 18:55:23.103492 5933 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7dd86ba9692c4130af502c2d4242c641 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7dd86ba9692c4130af502c2d4242c641" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 42453 } }
I20260811 18:55:23.103704 5933 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7dd86ba9692c4130af502c2d4242c641 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:55:23.103767 5933 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7dd86ba9692c4130af502c2d4242c641 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dd86ba9692c4130af502c2d4242c641, State: Initialized, Role: FOLLOWER
I20260811 18:55:23.104215 5933 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7dd86ba9692c4130af502c2d4242c641 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7dd86ba9692c4130af502c2d4242c641" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 42453 } }
I20260811 18:55:23.104367 5933 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7dd86ba9692c4130af502c2d4242c641 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:55:23.104439 5933 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7dd86ba9692c4130af502c2d4242c641 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:55:23.104544 5933 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7dd86ba9692c4130af502c2d4242c641 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:55:23.105455 5933 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7dd86ba9692c4130af502c2d4242c641 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7dd86ba9692c4130af502c2d4242c641" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 42453 } }
I20260811 18:55:23.105773 5933 leader_election.cc:302] T 00000000000000000000000000000000 P 7dd86ba9692c4130af502c2d4242c641 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7dd86ba9692c4130af502c2d4242c641; no voters:
I20260811 18:55:23.106477 5933 leader_election.cc:288] T 00000000000000000000000000000000 P 7dd86ba9692c4130af502c2d4242c641 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:55:23.106591 5945 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7dd86ba9692c4130af502c2d4242c641 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:55:23.106805 5945 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7dd86ba9692c4130af502c2d4242c641 [term 1 LEADER]: Becoming Leader. State: Replica: 7dd86ba9692c4130af502c2d4242c641, State: Running, Role: LEADER
I20260811 18:55:23.107103 5945 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7dd86ba9692c4130af502c2d4242c641 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7dd86ba9692c4130af502c2d4242c641" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 42453 } }
I20260811 18:55:23.107620 5933 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7dd86ba9692c4130af502c2d4242c641 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:55:23.108587 5948 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7dd86ba9692c4130af502c2d4242c641 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7dd86ba9692c4130af502c2d4242c641. Latest consensus state: current_term: 1 leader_uuid: "7dd86ba9692c4130af502c2d4242c641" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7dd86ba9692c4130af502c2d4242c641" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 42453 } } }
I20260811 18:55:23.108727 5948 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7dd86ba9692c4130af502c2d4242c641 [sys.catalog]: This master's current role is: LEADER
I20260811 18:55:23.109001 5946 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7dd86ba9692c4130af502c2d4242c641 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7dd86ba9692c4130af502c2d4242c641" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7dd86ba9692c4130af502c2d4242c641" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 42453 } } }
I20260811 18:55:23.109117 5946 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7dd86ba9692c4130af502c2d4242c641 [sys.catalog]: This master's current role is: LEADER
I20260811 18:55:23.109228 5968 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:55:23.112545 5968 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:55:23.118641 5968 catalog_manager.cc:1347] Generated new cluster ID: 2f5c6f4ab44e4c82a80dac076794f677
I20260811 18:55:23.118697 5968 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:55:23.136616 5968 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:55:23.137552 5968 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:55:23.144851 5968 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7dd86ba9692c4130af502c2d4242c641: Generated new TSK 0
I20260811 18:55:23.145541 5968 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260811 18:55:23.183346 5993 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260811 18:55:23.183516 5993 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 18:55:23.183879 5941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:23.158140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40976 (local address 127.27.180.190:42453)
0811 18:55:23.158244 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:23.158247 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:23.158529 (+ 282us) server_negotiation.cc:407] Connection header received
0811 18:55:23.158734 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:23.158736 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:23.158791 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:23.158871 (+ 80us) server_negotiation.cc:226] Negotiated authn=JWT
0811 18:55:23.159128 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:23.159706 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:23.160558 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:23.160830 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:23.160916 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:23.161268 (+ 352us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 18:55:23.183427 (+ 22159us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36585/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0811 18:55:23.183721 (+ 294us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:40976: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36585/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":35}
W20260811 18:55:23.183956 5941 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:40976: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36585/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260811 18:55:23.185465 5990 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:55:23.158099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:42453 (local address 127.0.0.1:40976)
0811 18:55:23.158448 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:23.158459 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:23.158583 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:23.158899 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:23.158902 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:23.158917 (+ 15us) client_negotiation.cc:189] Negotiated authn=JWT
0811 18:55:23.159030 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:23.159035 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:23.159808 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:23.159812 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:23.160452 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:23.160459 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:23.160935 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:23.160940 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:23.161031 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:23.161073 (+ 42us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 18:55:23.183756 (+ 22683us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36585/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0811 18:55:23.184012 (+ 256us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.27.180.190:42453: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36585/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":134,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.27.180.190:42453: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36585/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260811 18:55:23.186156 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 5799
2026-08-11T18:55:23Z chronyd exiting
I20260811 18:55:23.236373 28370 test_util.cc:175] -----------------------------------------------
I20260811 18:55:23.236459 28370 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786474496157106-28370-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (337 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260811 18:55:23.237669 28370 mini_oidc.cc:152] Starting JWKS server
I20260811 18:55:23.238517 28370 webserver.cc:460] Webserver started at https://127.0.0.1:43575/ using document root <none> and password file <none>
I20260811 18:55:23.238603 28370 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 18:55:23.238858 28370 webserver.cc:460] Webserver started at http://127.0.0.1:32861/ using document root <none> and password file <none>
2026-08-11T18:55:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:55:23Z Disabled control of system clock
I20260811 18:55:23.248548 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786474496157106-28370-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786474496157106-28370-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786474496157106-28370-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.190:41451
--webserver_interface=127.27.180.190
--webserver_port=0
--builtin_ntp_servers=127.27.180.148:37495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.180.190:41451
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43575/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786474496157106-28370-0/testchainca.cert with env {}
W20260811 18:55:23.357860 6005 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:55:23.358098 6005 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:55:23.358134 6005 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:55:23.361974 6005 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:55:23.362015 6005 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:55:23.362028 6005 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:55:23.362043 6005 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:55:23.362056 6005 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 18:55:23.362073 6005 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 18:55:23.362089 6005 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:43575/jwks.json
I20260811 18:55:23.367153 6005 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:37495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786474496157106-28370-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786474496157106-28370-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.180.190:41451
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.190:41451
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43575/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.180.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786474496157106-28370-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6005
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786474496157106-28370-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:55:23.368046 6005 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:55:23.368948 6005 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:55:23.376876 6017 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:55:23.376928 6005 server_base.cc:1034] running on GCE node
W20260811 18:55:23.376835 6012 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:23.376857 6015 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:55:23.377600 6005 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:55:23.378607 6005 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:55:23.379796 6005 hybrid_clock.cc:648] HybridClock initialized: now 1786474523379773 us; error 33 us; skew 500 ppm
I20260811 18:55:23.381618 6005 webserver.cc:460] Webserver started at http://127.27.180.190:44291/ using document root <none> and password file <none>
I20260811 18:55:23.382200 6005 fs_manager.cc:359] Metadata directory not provided
I20260811 18:55:23.382264 6005 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:55:23.382454 6005 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:55:23.384358 6005 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786474496157106-28370-0/minicluster-data/master-0/data/instance:
uuid: "27676413d527431e8a95464b52f9ae48"
format_stamp: "Formatted at 2026-08-11 18:55:23 on dist-test-slave-2j7r"
server_key: "2252ed3217d4d119288878471593c443"
server_key_iv: "d9adef55d88d0bbecb581fa30f636632"
server_key_version: "encryptionkey@0"
I20260811 18:55:23.384873 6005 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786474496157106-28370-0/minicluster-data/master-0/wal/instance:
uuid: "27676413d527431e8a95464b52f9ae48"
format_stamp: "Formatted at 2026-08-11 18:55:23 on dist-test-slave-2j7r"
server_key: "2252ed3217d4d119288878471593c443"
server_key_iv: "d9adef55d88d0bbecb581fa30f636632"
server_key_version: "encryptionkey@0"
I20260811 18:55:23.388259 6005 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260811 18:55:23.390518 6026 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:23.391677 6005 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 18:55:23.391790 6005 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786474496157106-28370-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786474496157106-28370-0/minicluster-data/master-0/wal
uuid: "27676413d527431e8a95464b52f9ae48"
format_stamp: "Formatted at 2026-08-11 18:55:23 on dist-test-slave-2j7r"
server_key: "2252ed3217d4d119288878471593c443"
server_key_iv: "d9adef55d88d0bbecb581fa30f636632"
server_key_version: "encryptionkey@0"
I20260811 18:55:23.391889 6005 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786474496157106-28370-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786474496157106-28370-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786474496157106-28370-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:55:23.411748 6005 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:55:23.412367 6005 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:55:23.412524 6005 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:55:23.419327 6005 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.190:41451
I20260811 18:55:23.419373 6131 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.190:41451 every 8 connection(s)
I20260811 18:55:23.420212 6005 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
I20260811 18:55:23.423334 6132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:55:23.423913 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 6005
I20260811 18:55:23.424015 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786474496157106-28370-0/minicluster-data/master-0/wal/instance
I20260811 18:55:23.424283 28370 external_mini_cluster.cc:1428] Setting key 0878c7183dfefb3302a2526d3fb9ee69
I20260811 18:55:23.429786 6132 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27676413d527431e8a95464b52f9ae48: Bootstrap starting.
I20260811 18:55:23.432215 6132 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 27676413d527431e8a95464b52f9ae48: Neither blocks nor log segments found. Creating new log.
I20260811 18:55:23.433069 6132 log.cc:824] T 00000000000000000000000000000000 P 27676413d527431e8a95464b52f9ae48: Log is configured to *not* fsync() on all Append() calls
I20260811 18:55:23.434242 6140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:23.425642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52624 (local address 127.27.180.190:41451)
0811 18:55:23.426335 (+ 693us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:23.426354 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:23.426397 (+ 43us) server_negotiation.cc:407] Connection header received
0811 18:55:23.427237 (+ 840us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:23.427259 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:23.427727 (+ 468us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:23.428105 (+ 378us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:23.428533 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:23.429964 (+ 1431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:23.430854 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:23.431515 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:23.431667 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:23.432355 (+ 688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:23.432361 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:23.432376 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:55:23.432842 (+ 466us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:23.432909 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:55:23.432937 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:23.433032 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:23.433168 (+ 136us) server_negotiation.cc:299] Negotiation successful
0811 18:55:23.433435 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":119,"threads_started":1}
I20260811 18:55:23.435045 6132 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27676413d527431e8a95464b52f9ae48: No bootstrap required, opened a new log
I20260811 18:55:23.436798 28370 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:55:23.438760 6132 raft_consensus.cc:348] T 00000000000000000000000000000000 P 27676413d527431e8a95464b52f9ae48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27676413d527431e8a95464b52f9ae48" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 41451 } }
I20260811 18:55:23.438956 6132 raft_consensus.cc:374] T 00000000000000000000000000000000 P 27676413d527431e8a95464b52f9ae48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:55:23.438993 6132 raft_consensus.cc:729] T 00000000000000000000000000000000 P 27676413d527431e8a95464b52f9ae48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27676413d527431e8a95464b52f9ae48, State: Initialized, Role: FOLLOWER
I20260811 18:55:23.439348 6132 consensus_queue.cc:260] T 00000000000000000000000000000000 P 27676413d527431e8a95464b52f9ae48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27676413d527431e8a95464b52f9ae48" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 41451 } }
I20260811 18:55:23.439462 6132 raft_consensus.cc:388] T 00000000000000000000000000000000 P 27676413d527431e8a95464b52f9ae48 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:55:23.439504 6132 raft_consensus.cc:482] T 00000000000000000000000000000000 P 27676413d527431e8a95464b52f9ae48 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:55:23.439569 6132 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 27676413d527431e8a95464b52f9ae48 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:55:23.440346 6132 raft_consensus.cc:504] T 00000000000000000000000000000000 P 27676413d527431e8a95464b52f9ae48 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27676413d527431e8a95464b52f9ae48" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 41451 } }
I20260811 18:55:23.440621 6132 leader_election.cc:302] T 00000000000000000000000000000000 P 27676413d527431e8a95464b52f9ae48 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 27676413d527431e8a95464b52f9ae48; no voters:
I20260811 18:55:23.440886 6132 leader_election.cc:288] T 00000000000000000000000000000000 P 27676413d527431e8a95464b52f9ae48 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:55:23.440992 6144 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 27676413d527431e8a95464b52f9ae48 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:55:23.441215 6144 raft_consensus.cc:686] T 00000000000000000000000000000000 P 27676413d527431e8a95464b52f9ae48 [term 1 LEADER]: Becoming Leader. State: Replica: 27676413d527431e8a95464b52f9ae48, State: Running, Role: LEADER
I20260811 18:55:23.441551 6144 consensus_queue.cc:237] T 00000000000000000000000000000000 P 27676413d527431e8a95464b52f9ae48 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27676413d527431e8a95464b52f9ae48" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 41451 } }
I20260811 18:55:23.441987 6132 sys_catalog.cc:564] T 00000000000000000000000000000000 P 27676413d527431e8a95464b52f9ae48 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:55:23.443161 6145 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27676413d527431e8a95464b52f9ae48 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "27676413d527431e8a95464b52f9ae48" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27676413d527431e8a95464b52f9ae48" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 41451 } } }
I20260811 18:55:23.443295 6145 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27676413d527431e8a95464b52f9ae48 [sys.catalog]: This master's current role is: LEADER
I20260811 18:55:23.443588 6150 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27676413d527431e8a95464b52f9ae48 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 27676413d527431e8a95464b52f9ae48. Latest consensus state: current_term: 1 leader_uuid: "27676413d527431e8a95464b52f9ae48" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27676413d527431e8a95464b52f9ae48" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 41451 } } }
I20260811 18:55:23.443679 6163 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:55:23.443681 6150 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27676413d527431e8a95464b52f9ae48 [sys.catalog]: This master's current role is: LEADER
I20260811 18:55:23.446779 6163 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:55:23.451967 6163 catalog_manager.cc:1347] Generated new cluster ID: d5438cededc34e82828f961605442b93
I20260811 18:55:23.452030 6163 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:55:23.464036 6163 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:55:23.464980 6163 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:55:23.470849 6163 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 27676413d527431e8a95464b52f9ae48: Generated new TSK 0
I20260811 18:55:23.471498 6163 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260811 18:55:23.501503 6190 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260811 18:55:23.501591 6190 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 18:55:23.502177 6140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:23.486680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52640 (local address 127.27.180.190:41451)
0811 18:55:23.486804 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:23.486809 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:23.487050 (+ 241us) server_negotiation.cc:407] Connection header received
0811 18:55:23.487201 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:23.487203 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:23.487249 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:23.487341 (+ 92us) server_negotiation.cc:226] Negotiated authn=JWT
0811 18:55:23.487574 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:23.488108 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:23.489365 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:23.489590 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:23.489667 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:23.489959 (+ 292us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 18:55:23.501855 (+ 11896us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43575/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0811 18:55:23.502066 (+ 211us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:52640: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43575/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":52}
W20260811 18:55:23.502233 6140 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:52640: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43575/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260811 18:55:23.502420 6187 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:55:23.486600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:41451 (local address 127.0.0.1:52640)
0811 18:55:23.486962 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:23.486977 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:23.487092 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:23.487353 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:23.487356 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:23.487364 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0811 18:55:23.487488 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:23.487494 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:23.488203 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:23.488207 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:23.489253 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:23.489264 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:23.489688 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:23.489692 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:23.489794 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:23.489823 (+ 29us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 18:55:23.502050 (+ 12227us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43575/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0811 18:55:23.502236 (+ 186us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.27.180.190:41451: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43575/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":118,"threads_started":1}
I20260811 18:55:23.502920 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 6005
2026-08-11T18:55:23Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (317 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260811 18:55:23.554816 28370 mini_oidc.cc:152] Starting JWKS server
I20260811 18:55:23.555647 28370 webserver.cc:460] Webserver started at https://127.0.0.1:45287/ using document root <none> and password file <none>
I20260811 18:55:23.555729 28370 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 18:55:23.555953 28370 webserver.cc:460] Webserver started at http://127.0.0.1:42837/ using document root <none> and password file <none>
2026-08-11T18:55:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:55:23Z Disabled control of system clock
I20260811 18:55:23.565323 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786474496157106-28370-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786474496157106-28370-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786474496157106-28370-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.190:33811
--webserver_interface=127.27.180.190
--webserver_port=0
--builtin_ntp_servers=127.27.180.148:38399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.180.190:33811
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45287/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260811 18:55:23.671813 6207 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:55:23.672068 6207 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:55:23.672122 6207 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:55:23.676211 6207 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:55:23.676268 6207 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:55:23.676290 6207 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:55:23.676309 6207 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:55:23.676329 6207 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 18:55:23.676357 6207 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 18:55:23.676384 6207 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:45287/jwks.json
I20260811 18:55:23.681147 6207 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:38399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786474496157106-28370-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786474496157106-28370-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.180.190:33811
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.190:33811
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45287/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.180.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6207
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786474496157106-28370-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:55:23.682183 6207 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:55:23.683566 6207 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:55:23.691567 6217 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:23.691567 6219 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:23.691569 6221 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:55:23.691766 6207 server_base.cc:1034] running on GCE node
I20260811 18:55:23.692365 6207 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:55:23.693498 6207 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:55:23.694685 6207 hybrid_clock.cc:648] HybridClock initialized: now 1786474523694648 us; error 37 us; skew 500 ppm
I20260811 18:55:23.696485 6207 webserver.cc:460] Webserver started at http://127.27.180.190:40981/ using document root <none> and password file <none>
I20260811 18:55:23.697057 6207 fs_manager.cc:359] Metadata directory not provided
I20260811 18:55:23.697165 6207 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:55:23.697392 6207 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:55:23.699425 6207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786474496157106-28370-0/minicluster-data/master-0/data/instance:
uuid: "6f19cebbfe86405f92e5e63f032b37b4"
format_stamp: "Formatted at 2026-08-11 18:55:23 on dist-test-slave-2j7r"
server_key: "d0259dc6b3cd1d2d78333d649f3115e4"
server_key_iv: "78ac9666ce15b811800d3e554d8deaec"
server_key_version: "encryptionkey@0"
I20260811 18:55:23.700037 6207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786474496157106-28370-0/minicluster-data/master-0/wal/instance:
uuid: "6f19cebbfe86405f92e5e63f032b37b4"
format_stamp: "Formatted at 2026-08-11 18:55:23 on dist-test-slave-2j7r"
server_key: "d0259dc6b3cd1d2d78333d649f3115e4"
server_key_iv: "78ac9666ce15b811800d3e554d8deaec"
server_key_version: "encryptionkey@0"
I20260811 18:55:23.706238 6207 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260811 18:55:23.708590 6234 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:23.710167 6207 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 18:55:23.710424 6207 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786474496157106-28370-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786474496157106-28370-0/minicluster-data/master-0/wal
uuid: "6f19cebbfe86405f92e5e63f032b37b4"
format_stamp: "Formatted at 2026-08-11 18:55:23 on dist-test-slave-2j7r"
server_key: "d0259dc6b3cd1d2d78333d649f3115e4"
server_key_iv: "78ac9666ce15b811800d3e554d8deaec"
server_key_version: "encryptionkey@0"
I20260811 18:55:23.710549 6207 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786474496157106-28370-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786474496157106-28370-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786474496157106-28370-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:55:23.733580 6207 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:55:23.734259 6207 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:55:23.734411 6207 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:55:23.741623 6207 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.190:33811
I20260811 18:55:23.741644 6350 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.190:33811 every 8 connection(s)
I20260811 18:55:23.742702 6207 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
I20260811 18:55:23.745688 6357 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:55:23.750689 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 6207
I20260811 18:55:23.750818 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786474496157106-28370-0/minicluster-data/master-0/wal/instance
I20260811 18:55:23.751111 28370 external_mini_cluster.cc:1428] Setting key fa0fb7ec99e737075219174eb51b3fce
I20260811 18:55:23.751322 6357 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f19cebbfe86405f92e5e63f032b37b4: Bootstrap starting.
I20260811 18:55:23.753830 6357 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6f19cebbfe86405f92e5e63f032b37b4: Neither blocks nor log segments found. Creating new log.
I20260811 18:55:23.754545 6357 log.cc:824] T 00000000000000000000000000000000 P 6f19cebbfe86405f92e5e63f032b37b4: Log is configured to *not* fsync() on all Append() calls
I20260811 18:55:23.756384 6357 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f19cebbfe86405f92e5e63f032b37b4: No bootstrap required, opened a new log
I20260811 18:55:23.760320 6357 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6f19cebbfe86405f92e5e63f032b37b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f19cebbfe86405f92e5e63f032b37b4" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 33811 } }
I20260811 18:55:23.760519 6357 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6f19cebbfe86405f92e5e63f032b37b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:55:23.760574 6357 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6f19cebbfe86405f92e5e63f032b37b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f19cebbfe86405f92e5e63f032b37b4, State: Initialized, Role: FOLLOWER
I20260811 18:55:23.760581 6364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:23.752474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50762 (local address 127.27.180.190:33811)
0811 18:55:23.753132 (+ 658us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:23.753148 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:23.753183 (+ 35us) server_negotiation.cc:407] Connection header received
0811 18:55:23.753852 (+ 669us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:23.753868 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:23.754292 (+ 424us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:23.754605 (+ 313us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:23.755108 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:23.756128 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:23.757318 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:23.757857 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:23.757976 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:23.758842 (+ 866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:23.758846 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:23.758858 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:55:23.759232 (+ 374us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:23.759321 (+ 89us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:55:23.759343 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:23.759415 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:23.759572 (+ 157us) server_negotiation.cc:299] Negotiation successful
0811 18:55:23.759828 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":117,"threads_started":1}
I20260811 18:55:23.761025 6357 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6f19cebbfe86405f92e5e63f032b37b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f19cebbfe86405f92e5e63f032b37b4" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 33811 } }
I20260811 18:55:23.761188 6357 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6f19cebbfe86405f92e5e63f032b37b4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:55:23.761255 6357 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6f19cebbfe86405f92e5e63f032b37b4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:55:23.761350 6357 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6f19cebbfe86405f92e5e63f032b37b4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:55:23.763413 28370 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:55:23.768501 6357 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6f19cebbfe86405f92e5e63f032b37b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f19cebbfe86405f92e5e63f032b37b4" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 33811 } }
I20260811 18:55:23.768822 6357 leader_election.cc:302] T 00000000000000000000000000000000 P 6f19cebbfe86405f92e5e63f032b37b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6f19cebbfe86405f92e5e63f032b37b4; no voters:
I20260811 18:55:23.769124 6357 leader_election.cc:288] T 00000000000000000000000000000000 P 6f19cebbfe86405f92e5e63f032b37b4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:55:23.769222 6372 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6f19cebbfe86405f92e5e63f032b37b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:55:23.769415 6372 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6f19cebbfe86405f92e5e63f032b37b4 [term 1 LEADER]: Becoming Leader. State: Replica: 6f19cebbfe86405f92e5e63f032b37b4, State: Running, Role: LEADER
I20260811 18:55:23.769688 6372 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6f19cebbfe86405f92e5e63f032b37b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f19cebbfe86405f92e5e63f032b37b4" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 33811 } }
I20260811 18:55:23.770407 6357 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6f19cebbfe86405f92e5e63f032b37b4 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:55:23.770876 6378 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f19cebbfe86405f92e5e63f032b37b4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6f19cebbfe86405f92e5e63f032b37b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f19cebbfe86405f92e5e63f032b37b4" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 33811 } } }
I20260811 18:55:23.770994 6378 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f19cebbfe86405f92e5e63f032b37b4 [sys.catalog]: This master's current role is: LEADER
I20260811 18:55:23.771771 6373 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f19cebbfe86405f92e5e63f032b37b4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6f19cebbfe86405f92e5e63f032b37b4. Latest consensus state: current_term: 1 leader_uuid: "6f19cebbfe86405f92e5e63f032b37b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f19cebbfe86405f92e5e63f032b37b4" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 33811 } } }
I20260811 18:55:23.771865 6373 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f19cebbfe86405f92e5e63f032b37b4 [sys.catalog]: This master's current role is: LEADER
I20260811 18:55:23.772143 6382 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:55:23.775154 6382 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:55:23.780385 6382 catalog_manager.cc:1347] Generated new cluster ID: ac1f325d5a6741e8bddcefa4a71af8e3
I20260811 18:55:23.780594 6382 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:55:23.794421 6382 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:55:23.795528 6382 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:55:23.810809 6382 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6f19cebbfe86405f92e5e63f032b37b4: Generated new TSK 0
I20260811 18:55:23.811482 6382 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:55:23.830610 6364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:23.825757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50766 (local address 127.27.180.190:33811)
0811 18:55:23.825871 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:23.825876 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:23.826068 (+ 192us) server_negotiation.cc:407] Connection header received
0811 18:55:23.826224 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:23.826226 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:23.826282 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:23.826376 (+ 94us) server_negotiation.cc:226] Negotiated authn=JWT
0811 18:55:23.826612 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:23.827309 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:23.828224 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:23.828465 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:23.828552 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:23.828870 (+ 318us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 18:55:23.830247 (+ 1377us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45287/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0811 18:55:23.830473 (+ 226us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:50766: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45287/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":39}
W20260811 18:55:23.830674 6364 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:50766: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45287/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260811 18:55:23.832633 6419 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:55:23.825692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33811 (local address 127.0.0.1:50766)
0811 18:55:23.825984 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:23.825996 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:23.826100 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:23.826386 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:23.826389 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:23.826397 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0811 18:55:23.826508 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:23.826513 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:23.827419 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:23.827423 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:23.828108 (+ 685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:23.828119 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:23.828579 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:23.828583 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:23.828664 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:23.828688 (+ 24us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 18:55:23.830417 (+ 1729us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45287/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0811 18:55:23.832460 (+ 2043us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.27.180.190:33811: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45287/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":207,"mutex_wait_us":1842,"thread_start_us":102,"threads_started":1}
I20260811 18:55:23.833184 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 6207
E20260811 18:55:23.853574 6423 webserver.cc:533] Webserver: sslize failed:
E20260811 18:55:23.853680 6423 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-11T18:55:23Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (336 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-11T18:55:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:55:23Z Disabled control of system clock
I20260811 18:55:23.900265 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786474496157106-28370-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786474496157106-28370-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786474496157106-28370-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.190:45555
--webserver_interface=127.27.180.190
--webserver_port=0
--builtin_ntp_servers=127.27.180.148:37835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.180.190:45555
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786474496157106-28370-0/testchainca.cert with env {}
W20260811 18:55:24.009342 6432 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:55:24.009616 6432 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:55:24.009658 6432 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:55:24.013767 6432 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:55:24.013829 6432 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:55:24.013845 6432 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:55:24.013860 6432 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:55:24.013877 6432 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 18:55:24.013907 6432 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 18:55:24.013931 6432 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260811 18:55:24.019342 6432 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:37835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786474496157106-28370-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786474496157106-28370-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.180.190:45555
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.190:45555
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.180.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786474496157106-28370-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6432
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786474496157106-28370-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:55:24.020426 6432 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:55:24.021510 6432 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:55:24.030045 6448 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:24.030048 6451 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:24.030048 6444 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:55:24.030103 6432 server_base.cc:1034] running on GCE node
I20260811 18:55:24.031083 6432 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:55:24.032146 6432 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:55:24.033301 6432 hybrid_clock.cc:648] HybridClock initialized: now 1786474524033279 us; error 32 us; skew 500 ppm
I20260811 18:55:24.035277 6432 webserver.cc:460] Webserver started at http://127.27.180.190:38249/ using document root <none> and password file <none>
I20260811 18:55:24.035821 6432 fs_manager.cc:359] Metadata directory not provided
I20260811 18:55:24.035883 6432 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:55:24.036079 6432 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:55:24.038169 6432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786474496157106-28370-0/minicluster-data/master-0/data/instance:
uuid: "189efb3f6b364647887087ffe06e4bce"
format_stamp: "Formatted at 2026-08-11 18:55:24 on dist-test-slave-2j7r"
server_key: "5d395716caeacf0e94997546926d1f14"
server_key_iv: "63f114a6153ae5b69d5481e601871a0f"
server_key_version: "encryptionkey@0"
I20260811 18:55:24.038753 6432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786474496157106-28370-0/minicluster-data/master-0/wal/instance:
uuid: "189efb3f6b364647887087ffe06e4bce"
format_stamp: "Formatted at 2026-08-11 18:55:24 on dist-test-slave-2j7r"
server_key: "5d395716caeacf0e94997546926d1f14"
server_key_iv: "63f114a6153ae5b69d5481e601871a0f"
server_key_version: "encryptionkey@0"
I20260811 18:55:24.042301 6432 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 18:55:24.044687 6463 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:24.046056 6432 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 18:55:24.046175 6432 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786474496157106-28370-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786474496157106-28370-0/minicluster-data/master-0/wal
uuid: "189efb3f6b364647887087ffe06e4bce"
format_stamp: "Formatted at 2026-08-11 18:55:24 on dist-test-slave-2j7r"
server_key: "5d395716caeacf0e94997546926d1f14"
server_key_iv: "63f114a6153ae5b69d5481e601871a0f"
server_key_version: "encryptionkey@0"
I20260811 18:55:24.046291 6432 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786474496157106-28370-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786474496157106-28370-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786474496157106-28370-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:55:24.065393 6432 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:55:24.066151 6432 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:55:24.066334 6432 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:55:24.073378 6432 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.190:45555
I20260811 18:55:24.073421 6565 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.190:45555 every 8 connection(s)
I20260811 18:55:24.074323 6432 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
I20260811 18:55:24.075954 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 6432
I20260811 18:55:24.076067 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786474496157106-28370-0/minicluster-data/master-0/wal/instance
I20260811 18:55:24.076346 28370 external_mini_cluster.cc:1428] Setting key 77137d3ce0c0e524beb35f6cb847353e
I20260811 18:55:24.077562 6566 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:55:24.084064 6566 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 189efb3f6b364647887087ffe06e4bce: Bootstrap starting.
I20260811 18:55:24.085999 6575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:24.077743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38124 (local address 127.27.180.190:45555)
0811 18:55:24.078335 (+ 592us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:24.078351 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:24.078386 (+ 35us) server_negotiation.cc:407] Connection header received
0811 18:55:24.079013 (+ 627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:24.079030 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:24.079410 (+ 380us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:24.079675 (+ 265us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:24.080183 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:24.081206 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:24.082458 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:24.082951 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:24.083067 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:24.084025 (+ 958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:24.084029 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:24.084041 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:55:24.084553 (+ 512us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:24.084609 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:55:24.084631 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:24.084714 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:24.084939 (+ 225us) server_negotiation.cc:299] Negotiation successful
0811 18:55:24.085280 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":258,"thread_start_us":117,"threads_started":1}
I20260811 18:55:24.086684 6566 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 189efb3f6b364647887087ffe06e4bce: Neither blocks nor log segments found. Creating new log.
I20260811 18:55:24.087527 6566 log.cc:824] T 00000000000000000000000000000000 P 189efb3f6b364647887087ffe06e4bce: Log is configured to *not* fsync() on all Append() calls
I20260811 18:55:24.089031 28370 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:55:24.089985 6566 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 189efb3f6b364647887087ffe06e4bce: No bootstrap required, opened a new log
I20260811 18:55:24.093556 6566 raft_consensus.cc:348] T 00000000000000000000000000000000 P 189efb3f6b364647887087ffe06e4bce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "189efb3f6b364647887087ffe06e4bce" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 45555 } }
I20260811 18:55:24.093770 6566 raft_consensus.cc:374] T 00000000000000000000000000000000 P 189efb3f6b364647887087ffe06e4bce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:55:24.093825 6566 raft_consensus.cc:729] T 00000000000000000000000000000000 P 189efb3f6b364647887087ffe06e4bce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 189efb3f6b364647887087ffe06e4bce, State: Initialized, Role: FOLLOWER
I20260811 18:55:24.094211 6566 consensus_queue.cc:260] T 00000000000000000000000000000000 P 189efb3f6b364647887087ffe06e4bce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "189efb3f6b364647887087ffe06e4bce" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 45555 } }
I20260811 18:55:24.094337 6566 raft_consensus.cc:388] T 00000000000000000000000000000000 P 189efb3f6b364647887087ffe06e4bce [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:55:24.094400 6566 raft_consensus.cc:482] T 00000000000000000000000000000000 P 189efb3f6b364647887087ffe06e4bce [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:55:24.094496 6566 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 189efb3f6b364647887087ffe06e4bce [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:55:24.095415 6566 raft_consensus.cc:504] T 00000000000000000000000000000000 P 189efb3f6b364647887087ffe06e4bce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "189efb3f6b364647887087ffe06e4bce" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 45555 } }
I20260811 18:55:24.095715 6566 leader_election.cc:302] T 00000000000000000000000000000000 P 189efb3f6b364647887087ffe06e4bce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 189efb3f6b364647887087ffe06e4bce; no voters:
I20260811 18:55:24.095979 6566 leader_election.cc:288] T 00000000000000000000000000000000 P 189efb3f6b364647887087ffe06e4bce [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:55:24.096101 6580 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 189efb3f6b364647887087ffe06e4bce [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:55:24.096316 6580 raft_consensus.cc:686] T 00000000000000000000000000000000 P 189efb3f6b364647887087ffe06e4bce [term 1 LEADER]: Becoming Leader. State: Replica: 189efb3f6b364647887087ffe06e4bce, State: Running, Role: LEADER
I20260811 18:55:24.096601 6580 consensus_queue.cc:237] T 00000000000000000000000000000000 P 189efb3f6b364647887087ffe06e4bce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "189efb3f6b364647887087ffe06e4bce" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 45555 } }
I20260811 18:55:24.097397 6566 sys_catalog.cc:564] T 00000000000000000000000000000000 P 189efb3f6b364647887087ffe06e4bce [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:55:24.098364 6581 sys_catalog.cc:455] T 00000000000000000000000000000000 P 189efb3f6b364647887087ffe06e4bce [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "189efb3f6b364647887087ffe06e4bce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "189efb3f6b364647887087ffe06e4bce" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 45555 } } }
I20260811 18:55:24.098348 6584 sys_catalog.cc:455] T 00000000000000000000000000000000 P 189efb3f6b364647887087ffe06e4bce [sys.catalog]: SysCatalogTable state changed. Reason: New leader 189efb3f6b364647887087ffe06e4bce. Latest consensus state: current_term: 1 leader_uuid: "189efb3f6b364647887087ffe06e4bce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "189efb3f6b364647887087ffe06e4bce" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 45555 } } }
I20260811 18:55:24.098505 6581 sys_catalog.cc:458] T 00000000000000000000000000000000 P 189efb3f6b364647887087ffe06e4bce [sys.catalog]: This master's current role is: LEADER
I20260811 18:55:24.098505 6584 sys_catalog.cc:458] T 00000000000000000000000000000000 P 189efb3f6b364647887087ffe06e4bce [sys.catalog]: This master's current role is: LEADER
I20260811 18:55:24.099000 6595 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:55:24.101827 6595 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:55:24.107316 6595 catalog_manager.cc:1347] Generated new cluster ID: a0d0181b927745a8a6d5a0f464db104c
I20260811 18:55:24.107388 6595 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:55:24.119246 6595 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:55:24.120636 6595 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:55:24.136837 6595 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 189efb3f6b364647887087ffe06e4bce: Generated new TSK 0
I20260811 18:55:24.137575 6595 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:55:25.046934 6575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:24.150951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38134 (local address 127.27.180.190:45555)
0811 18:55:24.151094 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:24.151099 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:24.151321 (+ 222us) server_negotiation.cc:407] Connection header received
0811 18:55:24.151454 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:24.151456 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:24.151508 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:24.151601 (+ 93us) server_negotiation.cc:226] Negotiated authn=JWT
0811 18:55:24.151838 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:24.152387 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:24.153617 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:24.153878 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:24.153967 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:24.154279 (+ 312us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 18:55:25.046540 (+892261us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0811 18:55:25.046805 (+ 265us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:38134: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":42}
W20260811 18:55:25.047013 6575 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:38134: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260811 18:55:25.047472 6627 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:55:24.150920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:45555 (local address 127.0.0.1:38134)
0811 18:55:24.151212 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:24.151227 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:24.151340 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:24.151608 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:24.151611 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:24.151619 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0811 18:55:24.151728 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:24.151735 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:24.152487 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:24.152491 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:24.153501 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:24.153512 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:24.153994 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:24.153998 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:24.154091 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:24.154119 (+ 28us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 18:55:25.046793 (+892674us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0811 18:55:25.046969 (+ 176us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.27.180.190:45555: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":103,"threads_started":1}
I20260811 18:55:25.048029 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 6432
2026-08-11T18:55:25Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (1184 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (129 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (123 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474496157106-28370-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:55:25 dist-test-slave-2j7r krb5kdc[6653](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:55:25 dist-test-slave-2j7r krb5kdc[6653](info): set up 2 sockets
Aug 11 18:55:25 dist-test-slave-2j7r krb5kdc[6653](info): commencing operation
krb5kdc: starting...
W20260811 18:55:26.696260 28370 mini_kdc.cc:121] Time spent starting KDC: real 1.368s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474496157106-28370-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474496157106-28370-0/krb5kdc/test-user.keytab.
Aug 11 18:55:26 dist-test-slave-2j7r krb5kdc[6653](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474526, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:55:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:55:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.180.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.180.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.180.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab.
Entry for principal kudu/127.27.180.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab.
I20260811 18:55:26.815382 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474496157106-28370-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474496157106-28370-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474496157106-28370-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.190:40907
--webserver_interface=127.27.180.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab
--principal=kudu/127.27.180.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:45611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.180.190:40907
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:55:26.924712 6677 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:55:26.924952 6677 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:55:26.924993 6677 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:55:26.928916 6677 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:55:26.928967 6677 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:55:26.928984 6677 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:55:26.929003 6677 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:55:26.929029 6677 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:55:26.934126 6677 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:45611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474496157106-28370-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474496157106-28370-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.180.190:40907
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab
--principal=kudu/127.27.180.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.190:40907
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.180.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6677
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474496157106-28370-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:55:26.935168 6677 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:55:26.936149 6677 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:55:26.944288 6688 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:26.944343 6685 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:26.944339 6686 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:55:26.944397 6677 server_base.cc:1034] running on GCE node
I20260811 18:55:26.945073 6677 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:55:26.946194 6677 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:55:26.947355 6677 hybrid_clock.cc:648] HybridClock initialized: now 1786474526947333 us; error 31 us; skew 500 ppm
Aug 11 18:55:26 dist-test-slave-2j7r krb5kdc[6653](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474526, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:55:26.950174 6677 init.cc:377] Logged in from keytab as kudu/127.27.180.190@KRBTEST.COM (short username kudu)
I20260811 18:55:26.951030 6677 webserver.cc:460] Webserver started at http://127.27.180.190:34949/ using document root <none> and password file <none>
I20260811 18:55:26.951555 6677 fs_manager.cc:359] Metadata directory not provided
I20260811 18:55:26.951620 6677 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:55:26.951807 6677 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:55:26.953863 6677 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474496157106-28370-0/minicluster-data/master-0/data/instance:
uuid: "2db5fd642b59498f8e75c61cbf0322e5"
format_stamp: "Formatted at 2026-08-11 18:55:26 on dist-test-slave-2j7r"
server_key: "789cbb793d8d4339c59b15c7c484844c"
server_key_iv: "47ab2b88e4deeb4116292de151fd75ad"
server_key_version: "encryptionkey@0"
I20260811 18:55:26.954392 6677 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474496157106-28370-0/minicluster-data/master-0/wal/instance:
uuid: "2db5fd642b59498f8e75c61cbf0322e5"
format_stamp: "Formatted at 2026-08-11 18:55:26 on dist-test-slave-2j7r"
server_key: "789cbb793d8d4339c59b15c7c484844c"
server_key_iv: "47ab2b88e4deeb4116292de151fd75ad"
server_key_version: "encryptionkey@0"
I20260811 18:55:26.958026 6677 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:55:26.960292 6697 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:26.961499 6677 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 18:55:26.961611 6677 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474496157106-28370-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474496157106-28370-0/minicluster-data/master-0/wal
uuid: "2db5fd642b59498f8e75c61cbf0322e5"
format_stamp: "Formatted at 2026-08-11 18:55:26 on dist-test-slave-2j7r"
server_key: "789cbb793d8d4339c59b15c7c484844c"
server_key_iv: "47ab2b88e4deeb4116292de151fd75ad"
server_key_version: "encryptionkey@0"
I20260811 18:55:26.961696 6677 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474496157106-28370-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474496157106-28370-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474496157106-28370-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:55:26.980548 6677 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:55:26.983180 6677 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:55:26.983351 6677 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:55:26.990449 6677 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.190:40907
I20260811 18:55:26.990464 6794 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.190:40907 every 8 connection(s)
I20260811 18:55:26.991385 6677 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
I20260811 18:55:26.994127 6795 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:55:26.999493 6795 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2db5fd642b59498f8e75c61cbf0322e5: Bootstrap starting.
I20260811 18:55:27.000928 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 6677
I20260811 18:55:27.001037 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474496157106-28370-0/minicluster-data/master-0/wal/instance
I20260811 18:55:27.001319 28370 external_mini_cluster.cc:1428] Setting key 52b6915317a76913efb13fedeeaeae66
I20260811 18:55:27.002005 6795 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2db5fd642b59498f8e75c61cbf0322e5: Neither blocks nor log segments found. Creating new log.
I20260811 18:55:27.002847 6795 log.cc:824] T 00000000000000000000000000000000 P 2db5fd642b59498f8e75c61cbf0322e5: Log is configured to *not* fsync() on all Append() calls
I20260811 18:55:27.004959 6795 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2db5fd642b59498f8e75c61cbf0322e5: No bootstrap required, opened a new log
I20260811 18:55:27.009023 6795 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2db5fd642b59498f8e75c61cbf0322e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2db5fd642b59498f8e75c61cbf0322e5" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 40907 } }
I20260811 18:55:27.009315 6795 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2db5fd642b59498f8e75c61cbf0322e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:55:27.009378 6795 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2db5fd642b59498f8e75c61cbf0322e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2db5fd642b59498f8e75c61cbf0322e5, State: Initialized, Role: FOLLOWER
I20260811 18:55:27.009809 6795 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2db5fd642b59498f8e75c61cbf0322e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2db5fd642b59498f8e75c61cbf0322e5" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 40907 } }
I20260811 18:55:27.009928 6795 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2db5fd642b59498f8e75c61cbf0322e5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:55:27.009985 6795 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2db5fd642b59498f8e75c61cbf0322e5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
Aug 11 18:55:27 dist-test-slave-2j7r krb5kdc[6653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474526, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM
I20260811 18:55:27.010076 6795 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2db5fd642b59498f8e75c61cbf0322e5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:55:27.011056 6795 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2db5fd642b59498f8e75c61cbf0322e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2db5fd642b59498f8e75c61cbf0322e5" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 40907 } }
I20260811 18:55:27.011385 6795 leader_election.cc:302] T 00000000000000000000000000000000 P 2db5fd642b59498f8e75c61cbf0322e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2db5fd642b59498f8e75c61cbf0322e5; no voters:
I20260811 18:55:27.011698 6795 leader_election.cc:288] T 00000000000000000000000000000000 P 2db5fd642b59498f8e75c61cbf0322e5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:55:27.011802 6806 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2db5fd642b59498f8e75c61cbf0322e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:55:27.012034 6806 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2db5fd642b59498f8e75c61cbf0322e5 [term 1 LEADER]: Becoming Leader. State: Replica: 2db5fd642b59498f8e75c61cbf0322e5, State: Running, Role: LEADER
I20260811 18:55:27.012382 6806 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2db5fd642b59498f8e75c61cbf0322e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2db5fd642b59498f8e75c61cbf0322e5" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 40907 } }
I20260811 18:55:27.012923 6795 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2db5fd642b59498f8e75c61cbf0322e5 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:55:27.014000 6813 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2db5fd642b59498f8e75c61cbf0322e5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2db5fd642b59498f8e75c61cbf0322e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2db5fd642b59498f8e75c61cbf0322e5" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 40907 } } }
I20260811 18:55:27.014120 6813 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2db5fd642b59498f8e75c61cbf0322e5 [sys.catalog]: This master's current role is: LEADER
I20260811 18:55:27.014452 6814 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2db5fd642b59498f8e75c61cbf0322e5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2db5fd642b59498f8e75c61cbf0322e5. Latest consensus state: current_term: 1 leader_uuid: "2db5fd642b59498f8e75c61cbf0322e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2db5fd642b59498f8e75c61cbf0322e5" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 40907 } } }
I20260811 18:55:27.014540 6814 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2db5fd642b59498f8e75c61cbf0322e5 [sys.catalog]: This master's current role is: LEADER
I20260811 18:55:27.015254 6826 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:55:27.016403 6799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:27.002370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40588 (local address 127.27.180.190:40907)
0811 18:55:27.002767 (+ 397us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:27.002782 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:27.002819 (+ 37us) server_negotiation.cc:407] Connection header received
0811 18:55:27.003533 (+ 714us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:27.003558 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:27.003887 (+ 329us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:27.004163 (+ 276us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:27.005142 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:27.006113 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:27.007249 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:27.007696 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:27.007805 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:27.010657 (+ 2852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:27.010662 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:27.010676 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:27.010703 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:27.012693 (+ 1990us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:27.013231 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:27.013237 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:27.013242 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:27.013392 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:27.013708 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:27.013712 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:27.013713 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:27.014241 (+ 528us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:27.015120 (+ 879us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:27.015291 (+ 171us) server_negotiation.cc:299] Negotiation successful
0811 18:55:27.015590 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":260,"thread_start_us":99,"threads_started":1}
I20260811 18:55:27.018311 6826 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:55:27.023569 6826 catalog_manager.cc:1347] Generated new cluster ID: d2babae0a6a247d99c112a980d0b83ca
I20260811 18:55:27.023622 6826 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:55:27.032610 6826 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:55:27.033550 6826 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:55:27.039849 6826 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2db5fd642b59498f8e75c61cbf0322e5: Generated new TSK 0
I20260811 18:55:27.040465 6826 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.180.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.180.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.180.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.129.keytab.
Entry for principal kudu/127.27.180.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.129.keytab.
I20260811 18:55:27.068542 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474496157106-28370-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474496157106-28370-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474496157106-28370-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474496157106-28370-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.180.129:0
--local_ip_for_outbound_sockets=127.27.180.129
--webserver_interface=127.27.180.129
--webserver_port=0
--tserver_master_addrs=127.27.180.190:40907
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.129.keytab
--principal=kudu/127.27.180.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:45611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:55:27.177490 6849 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:55:27.177706 6849 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:55:27.177747 6849 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:55:27.181623 6849 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:55:27.181670 6849 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:55:27.181738 6849 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.180.129
I20260811 18:55:27.186720 6849 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:45611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474496157106-28370-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474496157106-28370-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.129.keytab
--principal=kudu/127.27.180.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.180.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474496157106-28370-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.180.129
--webserver_port=0
--tserver_master_addrs=127.27.180.190:40907
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6849
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.180.129
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474496157106-28370-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:55:27.187712 6849 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:55:27.188665 6849 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:55:27.197960 6855 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:27.198011 6861 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:27.198011 6857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:55:27.198187 6849 server_base.cc:1034] running on GCE node
I20260811 18:55:27.198731 6849 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:55:27.199458 6849 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:55:27.200641 6849 hybrid_clock.cc:648] HybridClock initialized: now 1786474527200620 us; error 34 us; skew 500 ppm
Aug 11 18:55:27 dist-test-slave-2j7r krb5kdc[6653](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474527, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:55:27.203603 6849 init.cc:377] Logged in from keytab as kudu/127.27.180.129@KRBTEST.COM (short username kudu)
I20260811 18:55:27.204501 6849 webserver.cc:460] Webserver started at http://127.27.180.129:45747/ using document root <none> and password file <none>
I20260811 18:55:27.205062 6849 fs_manager.cc:359] Metadata directory not provided
I20260811 18:55:27.205166 6849 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:55:27.205355 6849 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:55:27.207397 6849 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474496157106-28370-0/minicluster-data/ts-0/data/instance:
uuid: "b6476444f8144407962fd8685fd2c995"
format_stamp: "Formatted at 2026-08-11 18:55:27 on dist-test-slave-2j7r"
server_key: "9cab9791122536b6fc0736574105803e"
server_key_iv: "21af968e3fa67482b300b3e5b0ad0e0e"
server_key_version: "encryptionkey@0"
I20260811 18:55:27.207916 6849 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474496157106-28370-0/minicluster-data/ts-0/wal/instance:
uuid: "b6476444f8144407962fd8685fd2c995"
format_stamp: "Formatted at 2026-08-11 18:55:27 on dist-test-slave-2j7r"
server_key: "9cab9791122536b6fc0736574105803e"
server_key_iv: "21af968e3fa67482b300b3e5b0ad0e0e"
server_key_version: "encryptionkey@0"
I20260811 18:55:27.211668 6849 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 18:55:27.214215 6874 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:27.215420 6849 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 18:55:27.215533 6849 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474496157106-28370-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474496157106-28370-0/minicluster-data/ts-0/wal
uuid: "b6476444f8144407962fd8685fd2c995"
format_stamp: "Formatted at 2026-08-11 18:55:27 on dist-test-slave-2j7r"
server_key: "9cab9791122536b6fc0736574105803e"
server_key_iv: "21af968e3fa67482b300b3e5b0ad0e0e"
server_key_version: "encryptionkey@0"
I20260811 18:55:27.215629 6849 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474496157106-28370-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474496157106-28370-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474496157106-28370-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:55:27.235033 6849 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:55:27.237828 6849 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:55:27.238001 6849 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:55:27.238566 6849 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:55:27.239368 6849 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:55:27.239416 6849 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:27.239459 6849 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:55:27.239487 6849 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:27.248276 6849 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.129:44749
I20260811 18:55:27.248385 7080 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.129:44749 every 8 connection(s)
I20260811 18:55:27.249173 6849 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474496157106-28370-0/minicluster-data/ts-0/data/info.pb
I20260811 18:55:27.254937 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 6849
I20260811 18:55:27.255048 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474496157106-28370-0/minicluster-data/ts-0/wal/instance
I20260811 18:55:27.255363 28370 external_mini_cluster.cc:1428] Setting key b681bdbb380f1c9cd62d1c7d6b2faa14
Aug 11 18:55:27 dist-test-slave-2j7r krb5kdc[6653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474527, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.129@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM
I20260811 18:55:27.263217 6799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:27.250907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.129:36835 (local address 127.27.180.190:40907)
0811 18:55:27.251044 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:27.251049 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:27.251840 (+ 791us) server_negotiation.cc:407] Connection header received
0811 18:55:27.253170 (+ 1330us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:27.253174 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:27.253222 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:27.253303 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:27.254381 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:27.254918 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:27.256399 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:27.256791 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:27.256919 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:27.259550 (+ 2631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:27.259555 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:27.259558 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:27.259590 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:27.261136 (+ 1546us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:27.261652 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:27.261656 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:27.261658 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:27.261784 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:27.262099 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:27.262103 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:27.262104 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:27.262347 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:27.262458 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:27.262902 (+ 444us) server_negotiation.cc:299] Negotiation successful
0811 18:55:27.263041 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":22,"server-negotiator.queue_time_us":50}
I20260811 18:55:27.263880 7083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:27.251243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:40907 (local address 127.27.180.129:36835)
0811 18:55:27.251688 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:27.251730 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:27.252956 (+ 1226us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:27.253462 (+ 506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:27.253468 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:27.253813 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:27.254263 (+ 450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:27.254277 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:27.255039 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:27.255043 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:27.256261 (+ 1218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:27.256273 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:27.256967 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:27.256972 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:27.257187 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:27.257765 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:27.257783 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:27.259366 (+ 1583us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:55:27.261261 (+ 1895us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:27.261266 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:27.261277 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:27.261534 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:27.261881 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:27.261883 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:27.261884 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:27.262008 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:27.262453 (+ 445us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:55:27.262458 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:55:27.262675 (+ 217us) client_negotiation.cc:769] Sending connection context
0811 18:55:27.262828 (+ 153us) client_negotiation.cc:240] Negotiation successful
0811 18:55:27.263070 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":124,"threads_started":1}
I20260811 18:55:27.265122 7081 heartbeater.cc:347] Connected to a master server at 127.27.180.190:40907
I20260811 18:55:27.265341 7081 heartbeater.cc:464] Registering TS with master...
I20260811 18:55:27.265852 7081 heartbeater.cc:511] Master 127.27.180.190:40907 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.27.180.130@KRBTEST.COM; defaulting to no policy
I20260811 18:55:27.267349 6720 ts_manager.cc:194] Registered new tserver with Master: b6476444f8144407962fd8685fd2c995 (127.27.180.129:44749)
I20260811 18:55:27.268483 6720 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.180.129@KRBTEST.COM'} at 127.27.180.129:36835
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.180.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.180.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.130.keytab.
Entry for principal kudu/127.27.180.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.130.keytab.
I20260811 18:55:27.283977 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474496157106-28370-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474496157106-28370-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474496157106-28370-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474496157106-28370-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.180.130:0
--local_ip_for_outbound_sockets=127.27.180.130
--webserver_interface=127.27.180.130
--webserver_port=0
--tserver_master_addrs=127.27.180.190:40907
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.130.keytab
--principal=kudu/127.27.180.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:45611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:55:27.392712 7088 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:55:27.392949 7088 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:55:27.392988 7088 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:55:27.397123 7088 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:55:27.397174 7088 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:55:27.397228 7088 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.180.130
I20260811 18:55:27.402226 7088 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:45611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474496157106-28370-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474496157106-28370-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.130.keytab
--principal=kudu/127.27.180.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.180.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474496157106-28370-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.180.130
--webserver_port=0
--tserver_master_addrs=127.27.180.190:40907
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7088
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.180.130
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474496157106-28370-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:55:27.403105 7088 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:55:27.403995 7088 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:55:27.413102 7098 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:27.413146 7107 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:55:27.413141 7088 server_base.cc:1034] running on GCE node
W20260811 18:55:27.413111 7104 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:55:27.413717 7088 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:55:27.414350 7088 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:55:27.415500 7088 hybrid_clock.cc:648] HybridClock initialized: now 1786474527415489 us; error 31 us; skew 500 ppm
Aug 11 18:55:27 dist-test-slave-2j7r krb5kdc[6653](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474527, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:55:27.418095 7088 init.cc:377] Logged in from keytab as kudu/127.27.180.130@KRBTEST.COM (short username kudu)
I20260811 18:55:27.418875 7088 webserver.cc:460] Webserver started at http://127.27.180.130:43485/ using document root <none> and password file <none>
I20260811 18:55:27.419422 7088 fs_manager.cc:359] Metadata directory not provided
I20260811 18:55:27.419481 7088 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:55:27.419643 7088 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:55:27.421620 7088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474496157106-28370-0/minicluster-data/ts-1/data/instance:
uuid: "3577c35ffe0246ea91375352623ff543"
format_stamp: "Formatted at 2026-08-11 18:55:27 on dist-test-slave-2j7r"
server_key: "fde16278eb6c0f29a4065a2cbe18ff80"
server_key_iv: "da04275fab06005c0aa5ad24281b6957"
server_key_version: "encryptionkey@0"
I20260811 18:55:27.422117 7088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474496157106-28370-0/minicluster-data/ts-1/wal/instance:
uuid: "3577c35ffe0246ea91375352623ff543"
format_stamp: "Formatted at 2026-08-11 18:55:27 on dist-test-slave-2j7r"
server_key: "fde16278eb6c0f29a4065a2cbe18ff80"
server_key_iv: "da04275fab06005c0aa5ad24281b6957"
server_key_version: "encryptionkey@0"
I20260811 18:55:27.425594 7088 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260811 18:55:27.427996 7121 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:27.429210 7088 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 18:55:27.429318 7088 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474496157106-28370-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474496157106-28370-0/minicluster-data/ts-1/wal
uuid: "3577c35ffe0246ea91375352623ff543"
format_stamp: "Formatted at 2026-08-11 18:55:27 on dist-test-slave-2j7r"
server_key: "fde16278eb6c0f29a4065a2cbe18ff80"
server_key_iv: "da04275fab06005c0aa5ad24281b6957"
server_key_version: "encryptionkey@0"
I20260811 18:55:27.429411 7088 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474496157106-28370-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474496157106-28370-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474496157106-28370-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:55:27.439708 7088 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:55:27.442600 7088 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:55:27.442766 7088 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:55:27.443312 7088 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:55:27.444089 7088 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:55:27.444136 7088 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:27.444186 7088 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:55:27.444216 7088 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:27.452653 7088 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.130:42785
I20260811 18:55:27.452708 7357 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.130:42785 every 8 connection(s)
I20260811 18:55:27.453511 7088 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474496157106-28370-0/minicluster-data/ts-1/data/info.pb
I20260811 18:55:27.459935 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 7088
I20260811 18:55:27.460052 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474496157106-28370-0/minicluster-data/ts-1/wal/instance
I20260811 18:55:27.460319 28370 external_mini_cluster.cc:1428] Setting key d7cb4852c14625038e2c70069432d5aa
Aug 11 18:55:27 dist-test-slave-2j7r krb5kdc[6653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474527, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.130@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM
I20260811 18:55:27.466858 6799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:27.455120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.130:51375 (local address 127.27.180.190:40907)
0811 18:55:27.455239 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:27.455242 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:27.455999 (+ 757us) server_negotiation.cc:407] Connection header received
0811 18:55:27.457348 (+ 1349us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:27.457351 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:27.457410 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:27.457485 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:27.458560 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:27.459082 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:27.460267 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:27.460519 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:27.460611 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:27.463305 (+ 2694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:27.463308 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:27.463310 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:27.463335 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:27.464799 (+ 1464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:27.465309 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:27.465312 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:27.465314 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:27.465423 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:27.465751 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:27.465754 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:27.465755 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:27.465997 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:27.466088 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:27.466569 (+ 481us) server_negotiation.cc:299] Negotiation successful
0811 18:55:27.466697 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 18:55:27.467590 7365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:27.455420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:40907 (local address 127.27.180.130:51375)
0811 18:55:27.455845 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:27.455887 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:27.457153 (+ 1266us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:27.457651 (+ 498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:27.457657 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:27.457990 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:27.458425 (+ 435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:27.458438 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:27.459197 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:27.459200 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:27.460155 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:27.460166 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:27.460659 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:27.460663 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:27.460858 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:27.461471 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:27.461486 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:27.463140 (+ 1654us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:55:27.464913 (+ 1773us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:27.464917 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:27.464928 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:27.465217 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:27.465509 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:27.465512 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:27.465514 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:27.465651 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:27.466097 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:55:27.466103 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:55:27.466329 (+ 226us) client_negotiation.cc:769] Sending connection context
0811 18:55:27.466487 (+ 158us) client_negotiation.cc:240] Negotiation successful
0811 18:55:27.466793 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":117,"threads_started":1}
I20260811 18:55:27.468652 7359 heartbeater.cc:347] Connected to a master server at 127.27.180.190:40907
I20260811 18:55:27.468901 7359 heartbeater.cc:464] Registering TS with master...
I20260811 18:55:27.469494 7359 heartbeater.cc:511] Master 127.27.180.190:40907 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.27.180.131@KRBTEST.COM; defaulting to no policy
I20260811 18:55:27.470584 6720 ts_manager.cc:194] Registered new tserver with Master: 3577c35ffe0246ea91375352623ff543 (127.27.180.130:42785)
I20260811 18:55:27.471189 6720 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.180.130@KRBTEST.COM'} at 127.27.180.130:51375
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.180.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.180.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.131.keytab.
Entry for principal kudu/127.27.180.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.131.keytab.
I20260811 18:55:27.487393 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474496157106-28370-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474496157106-28370-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474496157106-28370-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474496157106-28370-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.180.131:0
--local_ip_for_outbound_sockets=127.27.180.131
--webserver_interface=127.27.180.131
--webserver_port=0
--tserver_master_addrs=127.27.180.190:40907
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.131.keytab
--principal=kudu/127.27.180.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:45611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:55:27.596593 7374 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:55:27.596825 7374 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:55:27.596865 7374 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:55:27.600745 7374 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:55:27.600795 7374 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:55:27.600848 7374 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.180.131
I20260811 18:55:27.605944 7374 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:45611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474496157106-28370-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474496157106-28370-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.131.keytab
--principal=kudu/127.27.180.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.180.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474496157106-28370-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.180.131
--webserver_port=0
--tserver_master_addrs=127.27.180.190:40907
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7374
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.180.131
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474496157106-28370-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:55:27.606840 7374 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:55:27.607726 7374 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:55:27.616647 7389 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:55:27.616830 7374 server_base.cc:1034] running on GCE node
W20260811 18:55:27.616662 7386 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:27.616729 7384 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:55:27.617327 7374 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:55:27.617884 7374 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:55:27.619035 7374 hybrid_clock.cc:648] HybridClock initialized: now 1786474527619006 us; error 38 us; skew 500 ppm
Aug 11 18:55:27 dist-test-slave-2j7r krb5kdc[6653](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474527, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:55:27.621600 7374 init.cc:377] Logged in from keytab as kudu/127.27.180.131@KRBTEST.COM (short username kudu)
I20260811 18:55:27.622437 7374 webserver.cc:460] Webserver started at http://127.27.180.131:46205/ using document root <none> and password file <none>
I20260811 18:55:27.622931 7374 fs_manager.cc:359] Metadata directory not provided
I20260811 18:55:27.622988 7374 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:55:27.623174 7374 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:55:27.625054 7374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474496157106-28370-0/minicluster-data/ts-2/data/instance:
uuid: "fa46ee0335d84222ae9b3be275170d2e"
format_stamp: "Formatted at 2026-08-11 18:55:27 on dist-test-slave-2j7r"
server_key: "47a37a5453be0e96bd9646b7adc67ade"
server_key_iv: "a1de10ff28ab2293cea279eae81dec02"
server_key_version: "encryptionkey@0"
I20260811 18:55:27.625691 7374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474496157106-28370-0/minicluster-data/ts-2/wal/instance:
uuid: "fa46ee0335d84222ae9b3be275170d2e"
format_stamp: "Formatted at 2026-08-11 18:55:27 on dist-test-slave-2j7r"
server_key: "47a37a5453be0e96bd9646b7adc67ade"
server_key_iv: "a1de10ff28ab2293cea279eae81dec02"
server_key_version: "encryptionkey@0"
I20260811 18:55:27.629122 7374 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:55:27.631338 7405 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:27.632450 7374 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 18:55:27.632551 7374 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474496157106-28370-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474496157106-28370-0/minicluster-data/ts-2/wal
uuid: "fa46ee0335d84222ae9b3be275170d2e"
format_stamp: "Formatted at 2026-08-11 18:55:27 on dist-test-slave-2j7r"
server_key: "47a37a5453be0e96bd9646b7adc67ade"
server_key_iv: "a1de10ff28ab2293cea279eae81dec02"
server_key_version: "encryptionkey@0"
I20260811 18:55:27.632635 7374 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474496157106-28370-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474496157106-28370-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474496157106-28370-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:55:27.645179 7374 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:55:27.648015 7374 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:55:27.648176 7374 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:55:27.648689 7374 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:55:27.649494 7374 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:55:27.649540 7374 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:27.649581 7374 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:55:27.649611 7374 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:27.657541 7374 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.131:36481
I20260811 18:55:27.657600 7626 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.131:36481 every 8 connection(s)
I20260811 18:55:27.658362 7374 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474496157106-28370-0/minicluster-data/ts-2/data/info.pb
I20260811 18:55:27.662937 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 7374
I20260811 18:55:27.663038 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474496157106-28370-0/minicluster-data/ts-2/wal/instance
I20260811 18:55:27.663318 28370 external_mini_cluster.cc:1428] Setting key 6d89507e799424bc97bc6c9d87ec50f4
Aug 11 18:55:27 dist-test-slave-2j7r krb5kdc[6653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474527, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.131@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM
I20260811 18:55:27.671021 6799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:27.660204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.131:50931 (local address 127.27.180.190:40907)
0811 18:55:27.660369 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:27.660373 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:27.661156 (+ 783us) server_negotiation.cc:407] Connection header received
0811 18:55:27.662472 (+ 1316us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:27.662476 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:27.662523 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:27.662597 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:27.663758 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:27.664435 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:27.665684 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:27.665908 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:27.666004 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:27.668268 (+ 2264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:27.668272 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:27.668274 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:27.668299 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:27.669462 (+ 1163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:27.669891 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:27.669893 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:27.669895 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:27.669960 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:27.670203 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:27.670205 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:27.670206 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:27.670369 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:27.670441 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:27.670819 (+ 378us) server_negotiation.cc:299] Negotiation successful
0811 18:55:27.670909 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 18:55:27.671765 7630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:27.660579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:40907 (local address 127.27.180.131:50931)
0811 18:55:27.660998 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:27.661036 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:27.662294 (+ 1258us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:27.662747 (+ 453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:27.662754 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:27.663109 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:27.663618 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:27.663632 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:27.664567 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:27.664571 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:27.665583 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:27.665593 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:27.666071 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:27.666074 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:27.666241 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:27.666696 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:27.666712 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:27.668107 (+ 1395us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:55:27.669555 (+ 1448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:27.669560 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:27.669570 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:27.669813 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:27.670020 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:27.670022 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:27.670024 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:27.670141 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:27.670439 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:55:27.670444 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:55:27.670644 (+ 200us) client_negotiation.cc:769] Sending connection context
0811 18:55:27.670789 (+ 145us) client_negotiation.cc:240] Negotiation successful
0811 18:55:27.671016 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":114,"threads_started":1}
I20260811 18:55:27.672828 7627 heartbeater.cc:347] Connected to a master server at 127.27.180.190:40907
I20260811 18:55:27.673028 7627 heartbeater.cc:464] Registering TS with master...
I20260811 18:55:27.673493 7627 heartbeater.cc:511] Master 127.27.180.190:40907 requested a full tablet report, sending...
I20260811 18:55:27.674427 6720 ts_manager.cc:194] Registered new tserver with Master: fa46ee0335d84222ae9b3be275170d2e (127.27.180.131:36481)
I20260811 18:55:27.674947 6720 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.180.131@KRBTEST.COM'} at 127.27.180.131:50931
I20260811 18:55:27.676770 28370 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474496157106-28370-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474496157106-28370-0/krb5kdc/test-admin.keytab.
Aug 11 18:55:27 dist-test-slave-2j7r krb5kdc[6653](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474527, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:55:27.692987 28370 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260811 18:55:27.695436 7639 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:55:27.694279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42644 (local address 127.27.180.129:44749)
0811 18:55:27.694573 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:27.694583 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:27.694604 (+ 21us) server_negotiation.cc:407] Connection header received
0811 18:55:27.694651 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:27.694655 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:27.694818 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:27.694933 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:27.695281 (+ 348us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42644: BlockingRecv error: recv got EOF from 127.0.0.1:42644 (error 108)
Metrics: {"server-negotiator.queue_time_us":197,"thread_start_us":87,"threads_started":1}
W20260811 18:55:27.695561 28370 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474496157106-28370-0/krb5kdc/krb5cc' of type FILE
Aug 11 18:55:27 dist-test-slave-2j7r krb5kdc[6653](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474527, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:55:27.696506 28370 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 18:55:27 dist-test-slave-2j7r krb5kdc[6653](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474527, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:55:27.697350 28370 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 18:55:27 dist-test-slave-2j7r krb5kdc[6653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474527, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.180.129@KRBTEST.COM
I20260811 18:55:27.705840 7639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:27.698118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42658 (local address 127.27.180.129:44749)
0811 18:55:27.698250 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:27.698254 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:27.698289 (+ 35us) server_negotiation.cc:407] Connection header received
0811 18:55:27.698393 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:27.698395 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:27.698441 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:27.698502 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:27.698977 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:27.699744 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:27.700495 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:27.700836 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:27.700914 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:27.702942 (+ 2028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:27.702952 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:27.702964 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:27.702987 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:27.704295 (+ 1308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:27.704681 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:27.704685 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:27.704687 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:27.704764 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:27.704946 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:27.704948 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:27.704949 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:27.705221 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:27.705366 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:27.705538 (+ 172us) server_negotiation.cc:299] Negotiation successful
0811 18:55:27.705678 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
W20260811 18:55:27.709165 7639 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:55:27.708377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42674 (local address 127.27.180.129:44749)
0811 18:55:27.708507 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:27.708510 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:27.708571 (+ 61us) server_negotiation.cc:407] Connection header received
0811 18:55:27.708651 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:27.708653 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:27.708684 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:27.708741 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:27.709045 (+ 304us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42674: BlockingRecv error: recv got EOF from 127.0.0.1:42674 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260811 18:55:27.709288 28370 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474496157106-28370-0/krb5kdc/krb5cc' of type FILE
Aug 11 18:55:27 dist-test-slave-2j7r krb5kdc[6653](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474527, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:55:27.710156 28370 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 18:55:27 dist-test-slave-2j7r krb5kdc[6653](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474527, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:55:27.710929 28370 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 18:55:27 dist-test-slave-2j7r krb5kdc[6653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474527, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.180.129@KRBTEST.COM
I20260811 18:55:27.718713 7639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:27.711703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42678 (local address 127.27.180.129:44749)
0811 18:55:27.711812 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:27.711816 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:27.711880 (+ 64us) server_negotiation.cc:407] Connection header received
0811 18:55:27.712007 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:27.712010 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:27.712048 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:27.712105 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:27.712600 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:27.713121 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:27.713955 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:27.714168 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:27.714240 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:27.716102 (+ 1862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:27.716105 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:27.716106 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:27.716127 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:27.717340 (+ 1213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:27.717709 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:27.717712 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:27.717713 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:27.717783 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:27.718023 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:27.718026 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:27.718028 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:27.718172 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:27.718277 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:27.718499 (+ 222us) server_negotiation.cc:299] Negotiation successful
0811 18:55:27.718597 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
W20260811 18:55:27.720971 7639 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:55:27.720069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42690 (local address 127.27.180.129:44749)
0811 18:55:27.720161 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:27.720165 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:27.720354 (+ 189us) server_negotiation.cc:407] Connection header received
0811 18:55:27.720422 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:27.720424 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:27.720457 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:27.720512 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:27.720893 (+ 381us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42690: BlockingRecv error: recv got EOF from 127.0.0.1:42690 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
W20260811 18:55:27.721180 28370 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786474496157106-28370-0/krb5kdc/krb5cc' of type FILE
Aug 11 18:55:27 dist-test-slave-2j7r krb5kdc[6653](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474527, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:55:27.722021 28370 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 18:55:27 dist-test-slave-2j7r krb5kdc[6653](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474527, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:55:27.722826 28370 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 18:55:27 dist-test-slave-2j7r krb5kdc[6653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474527, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.180.129@KRBTEST.COM
I20260811 18:55:27.730111 7639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:27.723666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42698 (local address 127.27.180.129:44749)
0811 18:55:27.723764 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:27.723767 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:27.723911 (+ 144us) server_negotiation.cc:407] Connection header received
0811 18:55:27.723963 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:27.723964 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:27.723996 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:27.724054 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:27.724437 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:27.724873 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:27.725614 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:27.725809 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:27.725881 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:27.727658 (+ 1777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:27.727661 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:27.727662 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:27.727683 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:27.728883 (+ 1200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:27.729298 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:27.729301 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:27.729302 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:27.729369 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:27.729548 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:27.729550 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:27.729551 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:27.729683 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:27.729769 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:27.729919 (+ 150us) server_negotiation.cc:299] Negotiation successful
0811 18:55:27.730002 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 18:55:27.731002 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 6849
I20260811 18:55:27.745715 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 7088
I20260811 18:55:27.761534 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 7374
I20260811 18:55:27.777768 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 6677
2026-08-11T18:55:27Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2474 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:55:27 dist-test-slave-2j7r krb5kdc[7666](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:55:27 dist-test-slave-2j7r krb5kdc[7666](info): set up 2 sockets
Aug 11 18:55:27 dist-test-slave-2j7r krb5kdc[7666](info): commencing operation
krb5kdc: starting...
W20260811 18:55:28.967500 28370 mini_kdc.cc:121] Time spent starting KDC: real 1.165s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/krb5kdc/test-user.keytab.
Aug 11 18:55:29 dist-test-slave-2j7r krb5kdc[7666](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474529, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:55:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:55:29Z Disabled control of system clock
WARNING: no policy specified for oryx/127.27.180.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.27.180.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.27.180.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/krb5kdc/oryx_127.27.180.190.keytab.
Entry for principal oryx/127.27.180.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/krb5kdc/oryx_127.27.180.190.keytab.
I20260811 18:55:29.087466 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.190:46847
--webserver_interface=127.27.180.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/krb5kdc/oryx_127.27.180.190.keytab
--principal=oryx/127.27.180.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:33071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.180.190:46847
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:55:29.194334 7699 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:55:29.194566 7699 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:55:29.194603 7699 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:55:29.198470 7699 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:55:29.198519 7699 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:55:29.198537 7699 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:55:29.198555 7699 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:55:29.198583 7699 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:55:29.203781 7699 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:33071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.180.190:46847
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/krb5kdc/oryx_127.27.180.190.keytab
--principal=oryx/127.27.180.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.190:46847
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.180.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7699
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:55:29.204788 7699 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:55:29.205804 7699 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:55:29.213999 7707 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:29.213994 7714 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:29.213997 7708 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:55:29.214244 7699 server_base.cc:1034] running on GCE node
I20260811 18:55:29.214937 7699 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:55:29.216099 7699 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:55:29.217314 7699 hybrid_clock.cc:648] HybridClock initialized: now 1786474529217295 us; error 30 us; skew 500 ppm
Aug 11 18:55:29 dist-test-slave-2j7r krb5kdc[7666](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474529, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.180.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:55:29.220258 7699 init.cc:377] Logged in from keytab as oryx/127.27.180.190@KRBTEST.COM (short username oryx)
I20260811 18:55:29.221309 7699 webserver.cc:460] Webserver started at http://127.27.180.190:38849/ using document root <none> and password file <none>
I20260811 18:55:29.221904 7699 fs_manager.cc:359] Metadata directory not provided
I20260811 18:55:29.221978 7699 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:55:29.222213 7699 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:55:29.224377 7699 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/master-0/data/instance:
uuid: "fec531dd027244268997c383dda94895"
format_stamp: "Formatted at 2026-08-11 18:55:29 on dist-test-slave-2j7r"
server_key: "ce44b40ca5d7d21141fe65fe4483bf75"
server_key_iv: "8a51a47871cbd1eadf9706a9b792e26f"
server_key_version: "encryptionkey@0"
I20260811 18:55:29.224963 7699 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/master-0/wal/instance:
uuid: "fec531dd027244268997c383dda94895"
format_stamp: "Formatted at 2026-08-11 18:55:29 on dist-test-slave-2j7r"
server_key: "ce44b40ca5d7d21141fe65fe4483bf75"
server_key_iv: "8a51a47871cbd1eadf9706a9b792e26f"
server_key_version: "encryptionkey@0"
I20260811 18:55:29.228814 7699 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.003s
I20260811 18:55:29.231383 7730 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:29.232648 7699 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260811 18:55:29.232769 7699 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/master-0/wal
uuid: "fec531dd027244268997c383dda94895"
format_stamp: "Formatted at 2026-08-11 18:55:29 on dist-test-slave-2j7r"
server_key: "ce44b40ca5d7d21141fe65fe4483bf75"
server_key_iv: "8a51a47871cbd1eadf9706a9b792e26f"
server_key_version: "encryptionkey@0"
I20260811 18:55:29.232872 7699 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:55:29.249277 7699 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:55:29.252439 7699 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:55:29.252633 7699 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:55:29.259721 7699 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.190:46847
I20260811 18:55:29.260004 7824 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.190:46847 every 8 connection(s)
I20260811 18:55:29.260746 7699 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
I20260811 18:55:29.263156 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 7699
I20260811 18:55:29.263311 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/master-0/wal/instance
I20260811 18:55:29.263652 28370 external_mini_cluster.cc:1428] Setting key e46e9e268ffdf83b6bd44fd46ea9955f
I20260811 18:55:29.263855 7825 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:55:29.271153 7825 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fec531dd027244268997c383dda94895: Bootstrap starting.
Aug 11 18:55:29 dist-test-slave-2j7r krb5kdc[7666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474529, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM
I20260811 18:55:29.274036 7825 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fec531dd027244268997c383dda94895: Neither blocks nor log segments found. Creating new log.
I20260811 18:55:29.274951 7825 log.cc:824] T 00000000000000000000000000000000 P fec531dd027244268997c383dda94895: Log is configured to *not* fsync() on all Append() calls
I20260811 18:55:29.277434 7825 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fec531dd027244268997c383dda94895: No bootstrap required, opened a new log
I20260811 18:55:29.278448 7828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:29.265051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36102 (local address 127.27.180.190:46847)
0811 18:55:29.265518 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:29.265539 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:29.265575 (+ 36us) server_negotiation.cc:407] Connection header received
0811 18:55:29.266241 (+ 666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:29.266258 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:29.266590 (+ 332us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:29.266865 (+ 275us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:29.267779 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:29.268775 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:29.269963 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:29.270444 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:29.270561 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:29.273321 (+ 2760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:29.273329 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:29.273345 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:29.273383 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:29.275267 (+ 1884us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:29.275777 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:29.275782 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:29.275787 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:29.275932 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:29.276194 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:29.276196 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:29.276198 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:29.276741 (+ 543us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:29.276942 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:29.277153 (+ 211us) server_negotiation.cc:299] Negotiation successful
0811 18:55:29.277475 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":123,"threads_started":1}
I20260811 18:55:29.281208 7825 raft_consensus.cc:348] T 00000000000000000000000000000000 P fec531dd027244268997c383dda94895 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fec531dd027244268997c383dda94895" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 46847 } }
I20260811 18:55:29.281431 7825 raft_consensus.cc:374] T 00000000000000000000000000000000 P fec531dd027244268997c383dda94895 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:55:29.281473 7825 raft_consensus.cc:729] T 00000000000000000000000000000000 P fec531dd027244268997c383dda94895 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fec531dd027244268997c383dda94895, State: Initialized, Role: FOLLOWER
I20260811 18:55:29.281859 7825 consensus_queue.cc:260] T 00000000000000000000000000000000 P fec531dd027244268997c383dda94895 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fec531dd027244268997c383dda94895" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 46847 } }
I20260811 18:55:29.281989 7825 raft_consensus.cc:388] T 00000000000000000000000000000000 P fec531dd027244268997c383dda94895 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:55:29.282035 7825 raft_consensus.cc:482] T 00000000000000000000000000000000 P fec531dd027244268997c383dda94895 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:55:29.282107 7825 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fec531dd027244268997c383dda94895 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:55:29.282990 7825 raft_consensus.cc:504] T 00000000000000000000000000000000 P fec531dd027244268997c383dda94895 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fec531dd027244268997c383dda94895" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 46847 } }
I20260811 18:55:29.283283 7825 leader_election.cc:302] T 00000000000000000000000000000000 P fec531dd027244268997c383dda94895 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fec531dd027244268997c383dda94895; no voters:
I20260811 18:55:29.283546 7825 leader_election.cc:288] T 00000000000000000000000000000000 P fec531dd027244268997c383dda94895 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:55:29.283668 7831 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fec531dd027244268997c383dda94895 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:55:29.283876 7831 raft_consensus.cc:686] T 00000000000000000000000000000000 P fec531dd027244268997c383dda94895 [term 1 LEADER]: Becoming Leader. State: Replica: fec531dd027244268997c383dda94895, State: Running, Role: LEADER
I20260811 18:55:29.284241 7831 consensus_queue.cc:237] T 00000000000000000000000000000000 P fec531dd027244268997c383dda94895 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fec531dd027244268997c383dda94895" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 46847 } }
I20260811 18:55:29.284729 7825 sys_catalog.cc:564] T 00000000000000000000000000000000 P fec531dd027244268997c383dda94895 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:55:29.285933 7834 sys_catalog.cc:455] T 00000000000000000000000000000000 P fec531dd027244268997c383dda94895 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fec531dd027244268997c383dda94895" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fec531dd027244268997c383dda94895" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 46847 } } }
I20260811 18:55:29.286059 7834 sys_catalog.cc:458] T 00000000000000000000000000000000 P fec531dd027244268997c383dda94895 [sys.catalog]: This master's current role is: LEADER
I20260811 18:55:29.286584 7835 sys_catalog.cc:455] T 00000000000000000000000000000000 P fec531dd027244268997c383dda94895 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fec531dd027244268997c383dda94895. Latest consensus state: current_term: 1 leader_uuid: "fec531dd027244268997c383dda94895" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fec531dd027244268997c383dda94895" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 46847 } } }
I20260811 18:55:29.286805 7835 sys_catalog.cc:458] T 00000000000000000000000000000000 P fec531dd027244268997c383dda94895 [sys.catalog]: This master's current role is: LEADER
I20260811 18:55:29.287307 7844 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:55:29.290272 7844 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:55:29.295804 7844 catalog_manager.cc:1347] Generated new cluster ID: c733e261438841f4aa5d657b05032fa9
I20260811 18:55:29.295861 7844 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:55:29.306344 7844 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:55:29.307253 7844 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:55:29.314710 7844 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fec531dd027244268997c383dda94895: Generated new TSK 0
I20260811 18:55:29.315361 7844 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.27.180.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.27.180.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.27.180.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/krb5kdc/oryx_127.27.180.129.keytab.
Entry for principal oryx/127.27.180.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/krb5kdc/oryx_127.27.180.129.keytab.
I20260811 18:55:29.348762 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.180.129:0
--local_ip_for_outbound_sockets=127.27.180.129
--webserver_interface=127.27.180.129
--webserver_port=0
--tserver_master_addrs=127.27.180.190:46847
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/krb5kdc/oryx_127.27.180.129.keytab
--principal=oryx/127.27.180.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:33071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:55:29.457436 7873 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:55:29.457659 7873 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:55:29.457698 7873 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:55:29.461521 7873 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:55:29.461567 7873 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:55:29.461632 7873 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.180.129
I20260811 18:55:29.466657 7873 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:33071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/krb5kdc/oryx_127.27.180.129.keytab
--principal=oryx/127.27.180.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.180.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.180.129
--webserver_port=0
--tserver_master_addrs=127.27.180.190:46847
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7873
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.180.129
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:55:29.467615 7873 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:55:29.468557 7873 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:55:29.477818 7883 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:29.477845 7886 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:55:29.477993 7873 server_base.cc:1034] running on GCE node
W20260811 18:55:29.478112 7888 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:55:29.478511 7873 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:55:29.479118 7873 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:55:29.480274 7873 hybrid_clock.cc:648] HybridClock initialized: now 1786474529480259 us; error 31 us; skew 500 ppm
Aug 11 18:55:29 dist-test-slave-2j7r krb5kdc[7666](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474529, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.180.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:55:29.482987 7873 init.cc:377] Logged in from keytab as oryx/127.27.180.129@KRBTEST.COM (short username oryx)
I20260811 18:55:29.483793 7873 webserver.cc:460] Webserver started at http://127.27.180.129:35595/ using document root <none> and password file <none>
I20260811 18:55:29.484347 7873 fs_manager.cc:359] Metadata directory not provided
I20260811 18:55:29.484418 7873 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:55:29.484617 7873 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:55:29.486716 7873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/ts-0/data/instance:
uuid: "b1221343a5f8419ab3f58e8e328cc30c"
format_stamp: "Formatted at 2026-08-11 18:55:29 on dist-test-slave-2j7r"
server_key: "55dcc668bcb1e2524a3ff8107dd43d9e"
server_key_iv: "f1b2aba774940062536ef7ba941ebd88"
server_key_version: "encryptionkey@0"
I20260811 18:55:29.487272 7873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/ts-0/wal/instance:
uuid: "b1221343a5f8419ab3f58e8e328cc30c"
format_stamp: "Formatted at 2026-08-11 18:55:29 on dist-test-slave-2j7r"
server_key: "55dcc668bcb1e2524a3ff8107dd43d9e"
server_key_iv: "f1b2aba774940062536ef7ba941ebd88"
server_key_version: "encryptionkey@0"
I20260811 18:55:29.490935 7873 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 18:55:29.493299 7902 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:29.494482 7873 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 18:55:29.494580 7873 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/ts-0/wal
uuid: "b1221343a5f8419ab3f58e8e328cc30c"
format_stamp: "Formatted at 2026-08-11 18:55:29 on dist-test-slave-2j7r"
server_key: "55dcc668bcb1e2524a3ff8107dd43d9e"
server_key_iv: "f1b2aba774940062536ef7ba941ebd88"
server_key_version: "encryptionkey@0"
I20260811 18:55:29.494678 7873 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:55:29.505054 7873 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:55:29.507699 7873 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:55:29.507859 7873 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:55:29.508436 7873 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:55:29.509272 7873 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:55:29.509320 7873 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:29.509359 7873 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:55:29.509387 7873 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:29.517690 7873 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.129:32985
I20260811 18:55:29.517799 8108 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.129:32985 every 8 connection(s)
I20260811 18:55:29.518555 7873 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/ts-0/data/info.pb
I20260811 18:55:29.524479 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 7873
I20260811 18:55:29.524595 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/ts-0/wal/instance
I20260811 18:55:29.524876 28370 external_mini_cluster.cc:1428] Setting key 7ff6ec42969bc8786015d23a57fe17b4
Aug 11 18:55:29 dist-test-slave-2j7r krb5kdc[7666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474529, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.180.129@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM
I20260811 18:55:29.533259 7828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:29.520385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.129:55131 (local address 127.27.180.190:46847)
0811 18:55:29.520542 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:29.520546 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:29.521452 (+ 906us) server_negotiation.cc:407] Connection header received
0811 18:55:29.522843 (+ 1391us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:29.522847 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:29.522909 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:29.522996 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:29.524189 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:29.524782 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:29.526307 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:29.526608 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:29.526715 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:29.529292 (+ 2577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:29.529297 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:29.529300 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:29.529332 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:29.530813 (+ 1481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:29.531381 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:29.531385 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:29.531386 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:29.531485 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:29.531930 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:29.531935 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:29.531937 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:29.532193 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:29.532313 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:29.532924 (+ 611us) server_negotiation.cc:299] Negotiation successful
0811 18:55:29.533058 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 18:55:29.534155 8112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:29.520716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:46847 (local address 127.27.180.129:55131)
0811 18:55:29.521293 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:29.521339 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:29.522623 (+ 1284us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:29.523188 (+ 565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:29.523195 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:29.523561 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:29.524023 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:29.524038 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:29.524914 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:29.524917 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:29.526150 (+ 1233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:29.526163 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:29.526773 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:29.526777 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:29.526928 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:29.527533 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:29.527550 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:29.529119 (+ 1569us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:55:29.530930 (+ 1811us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:29.530935 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:29.530946 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:29.531229 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:29.531595 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:29.531598 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:29.531600 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:29.531771 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:29.532327 (+ 556us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:55:29.532334 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:55:29.532652 (+ 318us) client_negotiation.cc:769] Sending connection context
0811 18:55:29.532841 (+ 189us) client_negotiation.cc:240] Negotiation successful
0811 18:55:29.533179 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":131,"threads_started":1}
I20260811 18:55:29.535439 8109 heartbeater.cc:347] Connected to a master server at 127.27.180.190:46847
I20260811 18:55:29.535656 8109 heartbeater.cc:464] Registering TS with master...
I20260811 18:55:29.536108 8109 heartbeater.cc:511] Master 127.27.180.190:46847 requested a full tablet report, sending...
WARNING: no policy specified for oryx/127.27.180.130@KRBTEST.COM; defaulting to no policy
I20260811 18:55:29.537655 7768 ts_manager.cc:194] Registered new tserver with Master: b1221343a5f8419ab3f58e8e328cc30c (127.27.180.129:32985)
I20260811 18:55:29.538730 7768 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.27.180.129@KRBTEST.COM'} at 127.27.180.129:55131
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.27.180.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.27.180.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/krb5kdc/oryx_127.27.180.130.keytab.
Entry for principal oryx/127.27.180.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/krb5kdc/oryx_127.27.180.130.keytab.
I20260811 18:55:29.554402 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.180.130:0
--local_ip_for_outbound_sockets=127.27.180.130
--webserver_interface=127.27.180.130
--webserver_port=0
--tserver_master_addrs=127.27.180.190:46847
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/krb5kdc/oryx_127.27.180.130.keytab
--principal=oryx/127.27.180.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:33071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:55:29.661561 8116 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:55:29.661788 8116 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:55:29.661828 8116 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:55:29.665663 8116 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:55:29.665711 8116 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:55:29.665769 8116 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.180.130
I20260811 18:55:29.670722 8116 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:33071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/krb5kdc/oryx_127.27.180.130.keytab
--principal=oryx/127.27.180.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.180.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.180.130
--webserver_port=0
--tserver_master_addrs=127.27.180.190:46847
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8116
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.180.130
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:55:29.671592 8116 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:55:29.672500 8116 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:55:29.681684 8123 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:29.681733 8125 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:29.681842 8131 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:55:29.681890 8116 server_base.cc:1034] running on GCE node
I20260811 18:55:29.682334 8116 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:55:29.682921 8116 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:55:29.684092 8116 hybrid_clock.cc:648] HybridClock initialized: now 1786474529684077 us; error 28 us; skew 500 ppm
Aug 11 18:55:29 dist-test-slave-2j7r krb5kdc[7666](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474529, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.180.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:55:29.686852 8116 init.cc:377] Logged in from keytab as oryx/127.27.180.130@KRBTEST.COM (short username oryx)
I20260811 18:55:29.687644 8116 webserver.cc:460] Webserver started at http://127.27.180.130:33069/ using document root <none> and password file <none>
I20260811 18:55:29.688153 8116 fs_manager.cc:359] Metadata directory not provided
I20260811 18:55:29.688216 8116 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:55:29.688417 8116 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:55:29.690393 8116 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/ts-1/data/instance:
uuid: "485e9ca42b2a45aa9aeeb564daf0edda"
format_stamp: "Formatted at 2026-08-11 18:55:29 on dist-test-slave-2j7r"
server_key: "e1227394f9b3fa0d1993f35aaa5a82df"
server_key_iv: "38383799bfbbf3f8a0fe6bcabfa42ea1"
server_key_version: "encryptionkey@0"
I20260811 18:55:29.690928 8116 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/ts-1/wal/instance:
uuid: "485e9ca42b2a45aa9aeeb564daf0edda"
format_stamp: "Formatted at 2026-08-11 18:55:29 on dist-test-slave-2j7r"
server_key: "e1227394f9b3fa0d1993f35aaa5a82df"
server_key_iv: "38383799bfbbf3f8a0fe6bcabfa42ea1"
server_key_version: "encryptionkey@0"
I20260811 18:55:29.694365 8116 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 18:55:29.696547 8151 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:29.697714 8116 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 18:55:29.697819 8116 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/ts-1/wal
uuid: "485e9ca42b2a45aa9aeeb564daf0edda"
format_stamp: "Formatted at 2026-08-11 18:55:29 on dist-test-slave-2j7r"
server_key: "e1227394f9b3fa0d1993f35aaa5a82df"
server_key_iv: "38383799bfbbf3f8a0fe6bcabfa42ea1"
server_key_version: "encryptionkey@0"
I20260811 18:55:29.697933 8116 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:55:29.720407 8116 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:55:29.723388 8116 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:55:29.723563 8116 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:55:29.724138 8116 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:55:29.724927 8116 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:55:29.724973 8116 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:29.725028 8116 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:55:29.725056 8116 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:29.733381 8116 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.130:45897
I20260811 18:55:29.733433 8373 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.130:45897 every 8 connection(s)
I20260811 18:55:29.734222 8116 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/ts-1/data/info.pb
I20260811 18:55:29.740520 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 8116
I20260811 18:55:29.740621 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/ts-1/wal/instance
I20260811 18:55:29.740913 28370 external_mini_cluster.cc:1428] Setting key cb0859bed399d02733b9d9708070a8f5
Aug 11 18:55:29 dist-test-slave-2j7r krb5kdc[7666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474529, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.180.130@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM
I20260811 18:55:29.747347 7828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:29.735931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.130:58775 (local address 127.27.180.190:46847)
0811 18:55:29.736063 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:29.736067 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:29.736888 (+ 821us) server_negotiation.cc:407] Connection header received
0811 18:55:29.738302 (+ 1414us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:29.738305 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:29.738354 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:29.738426 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:29.739595 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:29.740121 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:29.741292 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:29.741516 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:29.741599 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:29.744012 (+ 2413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:29.744016 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:29.744019 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:29.744048 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:29.745366 (+ 1318us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:29.745885 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:29.745887 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:29.745888 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:29.745967 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:29.746293 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:29.746295 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:29.746296 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:29.746452 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:29.746522 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:29.747084 (+ 562us) server_negotiation.cc:299] Negotiation successful
0811 18:55:29.747184 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 18:55:29.748397 8376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:29.736283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:46847 (local address 127.27.180.130:58775)
0811 18:55:29.736717 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:29.736784 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:29.738077 (+ 1293us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:29.738581 (+ 504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:29.738588 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:29.738927 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:29.739455 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:29.739471 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:29.740207 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:29.740210 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:29.741173 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:29.741183 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:29.741659 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:29.741663 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:29.741814 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:29.742331 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:29.742351 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:29.743855 (+ 1504us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:55:29.745478 (+ 1623us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:29.745484 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:29.745498 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:29.745803 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:29.746038 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:29.746041 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:29.746042 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:29.746208 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:29.746541 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:55:29.746546 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:55:29.746837 (+ 291us) client_negotiation.cc:769] Sending connection context
0811 18:55:29.747065 (+ 228us) client_negotiation.cc:240] Negotiation successful
0811 18:55:29.747389 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":111,"threads_started":1}
I20260811 18:55:29.749536 8374 heartbeater.cc:347] Connected to a master server at 127.27.180.190:46847
I20260811 18:55:29.749742 8374 heartbeater.cc:464] Registering TS with master...
I20260811 18:55:29.750178 8374 heartbeater.cc:511] Master 127.27.180.190:46847 requested a full tablet report, sending...
I20260811 18:55:29.751187 7768 ts_manager.cc:194] Registered new tserver with Master: 485e9ca42b2a45aa9aeeb564daf0edda (127.27.180.130:45897)
WARNING: no policy specified for oryx/127.27.180.131@KRBTEST.COM; defaulting to no policy
I20260811 18:55:29.751719 7768 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.27.180.130@KRBTEST.COM'} at 127.27.180.130:58775
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.27.180.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.27.180.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/krb5kdc/oryx_127.27.180.131.keytab.
Entry for principal oryx/127.27.180.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/krb5kdc/oryx_127.27.180.131.keytab.
I20260811 18:55:29.768625 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.180.131:0
--local_ip_for_outbound_sockets=127.27.180.131
--webserver_interface=127.27.180.131
--webserver_port=0
--tserver_master_addrs=127.27.180.190:46847
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/krb5kdc/oryx_127.27.180.131.keytab
--principal=oryx/127.27.180.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:33071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:55:29.876255 8383 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:55:29.876477 8383 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:55:29.876533 8383 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:55:29.880854 8383 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:55:29.880913 8383 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:55:29.880991 8383 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.180.131
I20260811 18:55:29.886102 8383 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:33071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/krb5kdc/oryx_127.27.180.131.keytab
--principal=oryx/127.27.180.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.180.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.180.131
--webserver_port=0
--tserver_master_addrs=127.27.180.190:46847
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8383
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.180.131
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:55:29.887025 8383 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:55:29.887930 8383 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:55:29.896775 8394 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:29.896782 8400 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:29.896782 8395 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:55:29.897030 8383 server_base.cc:1034] running on GCE node
I20260811 18:55:29.897401 8383 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:55:29.898020 8383 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:55:29.899173 8383 hybrid_clock.cc:648] HybridClock initialized: now 1786474529899149 us; error 39 us; skew 500 ppm
Aug 11 18:55:29 dist-test-slave-2j7r krb5kdc[7666](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474529, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.180.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:55:29.901822 8383 init.cc:377] Logged in from keytab as oryx/127.27.180.131@KRBTEST.COM (short username oryx)
I20260811 18:55:29.902572 8383 webserver.cc:460] Webserver started at http://127.27.180.131:37389/ using document root <none> and password file <none>
I20260811 18:55:29.903044 8383 fs_manager.cc:359] Metadata directory not provided
I20260811 18:55:29.903101 8383 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:55:29.903267 8383 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:55:29.905143 8383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/ts-2/data/instance:
uuid: "93dbcfa6e0784096bed84ddb3f611598"
format_stamp: "Formatted at 2026-08-11 18:55:29 on dist-test-slave-2j7r"
server_key: "15df247d31229e753373e60c424bf69f"
server_key_iv: "21bca37456fbe576c9dc8680a4a72ff5"
server_key_version: "encryptionkey@0"
I20260811 18:55:29.905653 8383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/ts-2/wal/instance:
uuid: "93dbcfa6e0784096bed84ddb3f611598"
format_stamp: "Formatted at 2026-08-11 18:55:29 on dist-test-slave-2j7r"
server_key: "15df247d31229e753373e60c424bf69f"
server_key_iv: "21bca37456fbe576c9dc8680a4a72ff5"
server_key_version: "encryptionkey@0"
I20260811 18:55:29.909016 8383 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.002s
I20260811 18:55:29.911260 8413 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:29.912379 8383 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 18:55:29.912482 8383 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/ts-2/wal
uuid: "93dbcfa6e0784096bed84ddb3f611598"
format_stamp: "Formatted at 2026-08-11 18:55:29 on dist-test-slave-2j7r"
server_key: "15df247d31229e753373e60c424bf69f"
server_key_iv: "21bca37456fbe576c9dc8680a4a72ff5"
server_key_version: "encryptionkey@0"
I20260811 18:55:29.912569 8383 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:55:29.933300 8383 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:55:29.935959 8383 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:55:29.936126 8383 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:55:29.936707 8383 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:55:29.937606 8383 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:55:29.937654 8383 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:29.937692 8383 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:55:29.937714 8383 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:29.945991 8383 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.131:39783
I20260811 18:55:29.946019 8644 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.131:39783 every 8 connection(s)
I20260811 18:55:29.946815 8383 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/ts-2/data/info.pb
I20260811 18:55:29.954231 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 8383
I20260811 18:55:29.954342 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/ts-2/wal/instance
I20260811 18:55:29.954617 28370 external_mini_cluster.cc:1428] Setting key 3ff50e571b08b45f1959cc266861dcb5
Aug 11 18:55:29 dist-test-slave-2j7r krb5kdc[7666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474529, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.180.131@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM
I20260811 18:55:29.959625 7828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:29.948497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.131:53421 (local address 127.27.180.190:46847)
0811 18:55:29.948623 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:29.948626 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:29.949418 (+ 792us) server_negotiation.cc:407] Connection header received
0811 18:55:29.950770 (+ 1352us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:29.950778 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:29.950835 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:29.950909 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:29.952011 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:29.952537 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:29.953674 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:29.953891 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:29.953980 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:29.956398 (+ 2418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:29.956401 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:29.956410 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:29.956436 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:29.957725 (+ 1289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:29.958198 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:29.958201 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:29.958203 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:29.958281 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:29.958589 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:29.958592 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:29.958594 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:29.958771 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:29.958856 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:29.959339 (+ 483us) server_negotiation.cc:299] Negotiation successful
0811 18:55:29.959471 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 18:55:29.960311 8651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:29.948821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:46847 (local address 127.27.180.131:53421)
0811 18:55:29.949283 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:29.949323 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:29.950568 (+ 1245us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:29.951084 (+ 516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:29.951091 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:29.951422 (+ 331us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:29.951871 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:29.951884 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:29.952656 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:29.952659 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:29.953557 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:29.953567 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:29.954038 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:29.954043 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:29.954195 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:29.954697 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:29.954720 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:29.956247 (+ 1527us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:55:29.957832 (+ 1585us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:29.957838 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:29.957851 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:29.958105 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:29.958363 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:29.958366 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:29.958368 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:29.958501 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:29.958871 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:55:29.958878 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:55:29.959103 (+ 225us) client_negotiation.cc:769] Sending connection context
0811 18:55:29.959259 (+ 156us) client_negotiation.cc:240] Negotiation successful
0811 18:55:29.959556 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":107,"threads_started":1}
I20260811 18:55:29.961323 8647 heartbeater.cc:347] Connected to a master server at 127.27.180.190:46847
I20260811 18:55:29.961536 8647 heartbeater.cc:464] Registering TS with master...
I20260811 18:55:29.961968 8647 heartbeater.cc:511] Master 127.27.180.190:46847 requested a full tablet report, sending...
I20260811 18:55:29.962890 7768 ts_manager.cc:194] Registered new tserver with Master: 93dbcfa6e0784096bed84ddb3f611598 (127.27.180.131:39783)
I20260811 18:55:29.963442 7768 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.27.180.131@KRBTEST.COM'} at 127.27.180.131:53421
I20260811 18:55:29.968173 28370 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.27.180.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.27.180.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.27.180.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/krb5kdc/oryx_127.27.180.189.keytab.
Entry for principal oryx/127.27.180.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/krb5kdc/oryx_127.27.180.189.keytab.
I20260811 18:55:29.996300 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.189:45319
--webserver_interface=127.27.180.189
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/krb5kdc/oryx_127.27.180.189.keytab
--principal=oryx/127.27.180.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:33071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.180.190:46847,127.27.180.189:45319
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:55:30.106066 8659 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:55:30.106313 8659 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:55:30.106353 8659 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:55:30.110256 8659 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:55:30.110306 8659 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:55:30.110323 8659 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:55:30.110342 8659 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:55:30.110357 8659 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:55:30.115453 8659 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:33071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.27.180.190:46847,127.27.180.189:45319
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/krb5kdc/oryx_127.27.180.189.keytab
--principal=oryx/127.27.180.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.189:45319
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.27.180.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8659
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
W20260811 18:55:30.115620 8659 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.27.180.190:46847,127.27.180.189:45319'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260811 18:55:30.116457 8659 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:55:30.117410 8659 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:55:30.125375 8670 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:30.125391 8674 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:55:30.125452 8659 server_base.cc:1034] running on GCE node
W20260811 18:55:30.125543 8672 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:55:30.126175 8659 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:55:30.127187 8659 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:55:30.128356 8659 hybrid_clock.cc:648] HybridClock initialized: now 1786474530128332 us; error 38 us; skew 500 ppm
Aug 11 18:55:30 dist-test-slave-2j7r krb5kdc[7666](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474530, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.180.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:55:30.131033 8659 init.cc:377] Logged in from keytab as oryx/127.27.180.189@KRBTEST.COM (short username oryx)
I20260811 18:55:30.131873 8659 webserver.cc:460] Webserver started at http://127.27.180.189:34439/ using document root <none> and password file <none>
I20260811 18:55:30.132397 8659 fs_manager.cc:359] Metadata directory not provided
I20260811 18:55:30.132462 8659 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:55:30.132648 8659 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:55:30.134662 8659 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/master-1/data/instance:
uuid: "e3781302d19a4a8a9342e98ea95ddc93"
format_stamp: "Formatted at 2026-08-11 18:55:30 on dist-test-slave-2j7r"
server_key: "959ee01066efa93df4ad009cb06d50f1"
server_key_iv: "36c6f64bd507ef311d374d291468b95c"
server_key_version: "encryptionkey@0"
I20260811 18:55:30.135195 8659 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/master-1/wal/instance:
uuid: "e3781302d19a4a8a9342e98ea95ddc93"
format_stamp: "Formatted at 2026-08-11 18:55:30 on dist-test-slave-2j7r"
server_key: "959ee01066efa93df4ad009cb06d50f1"
server_key_iv: "36c6f64bd507ef311d374d291468b95c"
server_key_version: "encryptionkey@0"
I20260811 18:55:30.138687 8659 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.005s
I20260811 18:55:30.141047 8689 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:30.142202 8659 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 18:55:30.142315 8659 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/master-1/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/master-1/wal
uuid: "e3781302d19a4a8a9342e98ea95ddc93"
format_stamp: "Formatted at 2026-08-11 18:55:30 on dist-test-slave-2j7r"
server_key: "959ee01066efa93df4ad009cb06d50f1"
server_key_iv: "36c6f64bd507ef311d374d291468b95c"
server_key_version: "encryptionkey@0"
I20260811 18:55:30.142405 8659 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:55:30.153439 8659 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:55:30.156260 8659 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:55:30.156463 8659 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:55:30.162971 8659 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.189:45319
I20260811 18:55:30.163236 8787 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.189:45319 every 8 connection(s)
I20260811 18:55:30.163861 8659 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/master-1/data/info.pb
I20260811 18:55:30.166491 8794 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:55:30.170035 8794 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.180.190" port: 46847 } has no permanent_uuid. Determining permanent_uuid...
I20260811 18:55:30.172324 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 8659
Aug 11 18:55:30 dist-test-slave-2j7r krb5kdc[7666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474530, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.180.189@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM
Aug 11 18:55:30 dist-test-slave-2j7r krb5kdc[7666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474529, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.27.180.189@KRBTEST.COM
I20260811 18:55:30.183600 7828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:30.171795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36110 (local address 127.27.180.190:46847)
0811 18:55:30.171937 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:30.171941 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:30.172751 (+ 810us) server_negotiation.cc:407] Connection header received
0811 18:55:30.173858 (+ 1107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:30.173861 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:30.173911 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:30.174007 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:30.174996 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:30.175814 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:30.177137 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:30.177473 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:30.177571 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:30.180037 (+ 2466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:30.180040 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:30.180042 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:30.180066 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:30.181543 (+ 1477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:30.182064 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:30.182068 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:30.182070 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:30.182159 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:30.182510 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:30.182513 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:30.182514 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:30.182757 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:30.182855 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:30.183331 (+ 476us) server_negotiation.cc:299] Negotiation successful
0811 18:55:30.183440 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 18:55:30.184366 8805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:30.172135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:46847 (local address 127.0.0.1:36110)
0811 18:55:30.172584 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:30.172641 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:30.173658 (+ 1017us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:30.174041 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:30.174049 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:30.174489 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:30.174852 (+ 363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:30.174872 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:30.175929 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:30.175931 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:30.177002 (+ 1071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:30.177014 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:30.177618 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:30.177621 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:30.177797 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:30.178311 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:30.178333 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:30.179902 (+ 1569us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:55:30.181645 (+ 1743us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:30.181653 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:30.181661 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:30.181974 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:30.182240 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:30.182243 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:30.182245 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:30.182425 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:30.182859 (+ 434us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:55:30.182864 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:55:30.183102 (+ 238us) client_negotiation.cc:769] Sending connection context
0811 18:55:30.183257 (+ 155us) client_negotiation.cc:240] Negotiation successful
0811 18:55:30.183570 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"mutex_wait_us":3,"thread_start_us":112,"threads_started":1}
I20260811 18:55:30.184369 8807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:30.173163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44400 (local address 127.27.180.189:45319)
0811 18:55:30.173483 (+ 320us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:30.173492 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:30.173525 (+ 33us) server_negotiation.cc:407] Connection header received
0811 18:55:30.173710 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:30.173723 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:30.173944 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:30.174110 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:30.174921 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:30.176095 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:30.177201 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:30.177777 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:30.177855 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:30.180405 (+ 2550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:30.180411 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:30.180420 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:30.180441 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:30.182013 (+ 1572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:30.182434 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:30.182439 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:30.182440 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:30.182526 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:30.182775 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:30.182778 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:30.182780 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:30.183112 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:30.183256 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:30.183501 (+ 245us) server_negotiation.cc:299] Negotiation successful
0811 18:55:30.183632 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":60,"server-negotiator.queue_time_us":223,"spinlock_wait_cycles":6400,"thread_start_us":139,"threads_started":1}
I20260811 18:55:30.185876 8794 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.180.189" port: 45319 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 18:55:30 dist-test-slave-2j7r krb5kdc[7666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474530, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.180.189@KRBTEST.COM for oryx/127.27.180.189@KRBTEST.COM
I20260811 18:55:30.192448 8805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:30.186347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.189:45319 (local address 127.0.0.1:44414)
0811 18:55:30.186433 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:30.186444 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:30.186501 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:30.186749 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:30.186751 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:30.186889 (+ 138us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:30.186965 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:30.186970 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:30.187676 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:30.187679 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:30.188170 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:30.188177 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:30.188595 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:30.188598 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:30.188709 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:30.189025 (+ 316us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:30.189035 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:30.190143 (+ 1108us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:55:30.191410 (+ 1267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:30.191413 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:30.191414 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:30.191585 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:30.191767 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:30.191768 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:30.191770 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:30.191829 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:30.192081 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:55:30.192083 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:55:30.192147 (+ 64us) client_negotiation.cc:769] Sending connection context
0811 18:55:30.192217 (+ 70us) client_negotiation.cc:240] Negotiation successful
0811 18:55:30.192309 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29,"mutex_wait_us":17}
I20260811 18:55:30.192468 8807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:30.186502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44414 (local address 127.27.180.189:45319)
0811 18:55:30.186601 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:30.186605 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:30.186615 (+ 10us) server_negotiation.cc:407] Connection header received
0811 18:55:30.186639 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:30.186640 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:30.186668 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:30.186726 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:30.187057 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:30.187567 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:30.188266 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:30.188505 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:30.188588 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:30.190254 (+ 1666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:30.190257 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:30.190259 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:30.190283 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:30.191312 (+ 1029us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:30.191650 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:30.191652 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:30.191653 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:30.191710 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:30.191885 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:30.191887 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:30.191888 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:30.192021 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:30.192087 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:30.192254 (+ 167us) server_negotiation.cc:299] Negotiation successful
0811 18:55:30.192355 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":22,"server-negotiator.queue_time_us":40}
I20260811 18:55:30.195246 8794 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3781302d19a4a8a9342e98ea95ddc93: Bootstrap starting.
I20260811 18:55:30.197768 8794 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e3781302d19a4a8a9342e98ea95ddc93: Neither blocks nor log segments found. Creating new log.
I20260811 18:55:30.198693 8794 log.cc:824] T 00000000000000000000000000000000 P e3781302d19a4a8a9342e98ea95ddc93: Log is configured to *not* fsync() on all Append() calls
I20260811 18:55:30.204046 8794 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3781302d19a4a8a9342e98ea95ddc93: No bootstrap required, opened a new log
I20260811 18:55:30.207161 8794 raft_consensus.cc:348] T 00000000000000000000000000000000 P e3781302d19a4a8a9342e98ea95ddc93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fec531dd027244268997c383dda94895" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 46847 } } peers { permanent_uuid: "e3781302d19a4a8a9342e98ea95ddc93" member_type: VOTER last_known_addr { host: "127.27.180.189" port: 45319 } }
I20260811 18:55:30.207365 8794 raft_consensus.cc:374] T 00000000000000000000000000000000 P e3781302d19a4a8a9342e98ea95ddc93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:55:30.207429 8794 raft_consensus.cc:729] T 00000000000000000000000000000000 P e3781302d19a4a8a9342e98ea95ddc93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3781302d19a4a8a9342e98ea95ddc93, State: Initialized, Role: FOLLOWER
I20260811 18:55:30.207856 8794 consensus_queue.cc:260] T 00000000000000000000000000000000 P e3781302d19a4a8a9342e98ea95ddc93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fec531dd027244268997c383dda94895" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 46847 } } peers { permanent_uuid: "e3781302d19a4a8a9342e98ea95ddc93" member_type: VOTER last_known_addr { host: "127.27.180.189" port: 45319 } }
I20260811 18:55:30.208361 8812 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3781302d19a4a8a9342e98ea95ddc93 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fec531dd027244268997c383dda94895" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 46847 } } peers { permanent_uuid: "e3781302d19a4a8a9342e98ea95ddc93" member_type: VOTER last_known_addr { host: "127.27.180.189" port: 45319 } } }
I20260811 18:55:30.208503 8812 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3781302d19a4a8a9342e98ea95ddc93 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:55:30.209043 8794 sys_catalog.cc:564] T 00000000000000000000000000000000 P e3781302d19a4a8a9342e98ea95ddc93 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 18:55:30.214473 8834 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e3781302d19a4a8a9342e98ea95ddc93: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:55:30.214560 8834 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:55:30.216041 8659 master_runner.cc:428] Detected that this master e3781302d19a4a8a9342e98ea95ddc93 is joining an existing cluster
I20260811 18:55:30.216102 8659 master_runner.cc:432] Initiating AddMaster RPC to add 127.27.180.189:45319
I20260811 18:55:30.227303 8807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:30.218501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44420 (local address 127.27.180.189:45319)
0811 18:55:30.218656 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:30.218660 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:30.218804 (+ 144us) server_negotiation.cc:407] Connection header received
0811 18:55:30.218972 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:30.218975 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:30.219016 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:30.219060 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:30.219677 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:30.220442 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:30.221637 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:30.221988 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:30.222061 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:30.223478 (+ 1417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:30.223482 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:30.223485 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:30.223510 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:30.225008 (+ 1498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:30.225549 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:30.225553 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:30.225554 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:30.225639 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:30.226207 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:30.226210 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:30.226211 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:30.226637 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:30.226711 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:30.227021 (+ 310us) server_negotiation.cc:299] Negotiation successful
0811 18:55:30.227136 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":78}
I20260811 18:55:30.227773 8842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:30.218446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.189:45319 (local address 127.0.0.1:44420)
0811 18:55:30.218728 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:30.218743 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:30.218866 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:30.219201 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:30.219204 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:30.219422 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:30.219562 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:30.219570 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:30.220560 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:30.220563 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:30.221515 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:30.221527 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:30.222210 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:30.222213 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:30.222375 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:30.222876 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:30.222891 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:30.223324 (+ 433us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:55:30.225146 (+ 1822us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:30.225149 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:30.225151 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:30.225444 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:30.225740 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:30.225743 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:30.225745 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:30.226111 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:30.226768 (+ 657us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:55:30.226771 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:55:30.226857 (+ 86us) client_negotiation.cc:769] Sending connection context
0811 18:55:30.227482 (+ 625us) client_negotiation.cc:240] Negotiation successful
0811 18:55:30.227606 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"mutex_wait_us":28,"thread_start_us":124,"threads_started":1}
I20260811 18:55:30.230516 7828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:30.218944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36116 (local address 127.27.180.190:46847)
0811 18:55:30.219075 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:30.219080 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:30.219597 (+ 517us) server_negotiation.cc:407] Connection header received
0811 18:55:30.219765 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:30.219769 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:30.219817 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:30.219866 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:30.220486 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:30.221319 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:30.222569 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:30.222915 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:30.222986 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:30.225511 (+ 2525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:30.225515 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:30.225518 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:30.225546 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:30.227245 (+ 1699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:30.229010 (+ 1765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:30.229013 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:30.229015 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:30.229135 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:30.229529 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:30.229533 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:30.229535 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:30.229755 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:30.229830 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:30.230209 (+ 379us) server_negotiation.cc:299] Negotiation successful
0811 18:55:30.230352 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 18:55:30.231226 8845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:30.219164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:46847 (local address 127.0.0.1:36116)
0811 18:55:30.219513 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:30.219525 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:30.219664 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:30.220007 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:30.220010 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:30.220212 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:30.220354 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:30.220360 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:30.221464 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:30.221467 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:30.222433 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:30.222444 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:30.223138 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:30.223142 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:30.223235 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:30.223786 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:30.223799 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:30.225366 (+ 1567us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:55:30.227728 (+ 2362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:30.227731 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:30.227733 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:30.228893 (+ 1160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:30.229332 (+ 439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:30.229335 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:30.229337 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:30.229427 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:30.229906 (+ 479us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:55:30.229909 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:55:30.230019 (+ 110us) client_negotiation.cc:769] Sending connection context
0811 18:55:30.230892 (+ 873us) client_negotiation.cc:240] Negotiation successful
0811 18:55:30.231041 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"mutex_wait_us":26,"thread_start_us":105,"threads_started":1}
Aug 11 18:55:30 dist-test-slave-2j7r krb5kdc[7666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474529, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.180.190@KRBTEST.COM for oryx/127.27.180.189@KRBTEST.COM
WARNING: no policy specified for oryx/127.27.180.188@KRBTEST.COM; defaulting to no policy
I20260811 18:55:30.251580 8846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:30.236086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.189:45319 (local address 127.0.0.1:44434)
0811 18:55:30.241397 (+ 5311us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:30.241458 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:30.241633 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:30.241994 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:30.242001 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:30.242460 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:30.242599 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:30.242606 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:30.243648 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:30.243651 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:30.244660 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:30.244671 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:30.245314 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:30.245318 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:30.245440 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:30.245996 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:30.246011 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:30.247888 (+ 1877us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:55:30.249747 (+ 1859us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:30.249758 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:30.249766 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:30.250083 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:30.250360 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:30.250363 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:30.250365 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:30.250468 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:30.250887 (+ 419us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:55:30.250894 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:55:30.251027 (+ 133us) client_negotiation.cc:769] Sending connection context
0811 18:55:30.251188 (+ 161us) client_negotiation.cc:240] Negotiation successful
0811 18:55:30.251396 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5120,"thread_start_us":103,"threads_started":1}
I20260811 18:55:30.251585 8807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:30.236109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44434 (local address 127.27.180.189:45319)
0811 18:55:30.236219 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:30.236224 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:30.241529 (+ 5305us) server_negotiation.cc:407] Connection header received
0811 18:55:30.241783 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:30.241787 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:30.241846 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:30.241942 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:30.242728 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:30.243534 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:30.244788 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:30.245191 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:30.245303 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:30.248031 (+ 2728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:30.248036 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:30.248038 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:30.248065 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:30.249630 (+ 1565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:30.250180 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:30.250183 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:30.250185 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:30.250267 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:30.250558 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:30.250561 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:30.250563 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:30.250765 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:30.250857 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:30.251313 (+ 456us) server_negotiation.cc:299] Negotiation successful
0811 18:55:30.251431 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 18:55:30.253713 7768 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.27.180.189:45319: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "e3781302d19a4a8a9342e98ea95ddc93"
member_type: NON_VOTER
last_known_addr {
host: "127.27.180.189"
port: 45319
}
attrs {
promote: true
}
}
dest_uuid: "fec531dd027244268997c383dda94895"
cas_config_opid_index: -1
I20260811 18:55:30.254612 7768 consensus_queue.cc:237] T 00000000000000000000000000000000 P fec531dd027244268997c383dda94895 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "fec531dd027244268997c383dda94895" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 46847 } } peers { permanent_uuid: "e3781302d19a4a8a9342e98ea95ddc93" member_type: NON_VOTER last_known_addr { host: "127.27.180.189" port: 45319 } attrs { promote: true } }
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.27.180.188@KRBTEST.COM" created.
I20260811 18:55:30.257263 8851 raft_consensus.cc:2932] T 00000000000000000000000000000000 P fec531dd027244268997c383dda94895 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER e3781302d19a4a8a9342e98ea95ddc93 (127.27.180.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "fec531dd027244268997c383dda94895" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 46847 } } peers { permanent_uuid: "e3781302d19a4a8a9342e98ea95ddc93" member_type: NON_VOTER last_known_addr { host: "127.27.180.189" port: 45319 } attrs { promote: true } } }
I20260811 18:55:30.258976 8851 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.27.180.189:45319
I20260811 18:55:30.259303 8743 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e3781302d19a4a8a9342e98ea95ddc93 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:55:30.260654 8743 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e3781302d19a4a8a9342e98ea95ddc93 [term 1 FOLLOWER]: Refusing update from remote peer fec531dd027244268997c383dda94895: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260811 18:55:30.267055 8855 consensus_queue.cc:1035] T 00000000000000000000000000000000 P fec531dd027244268997c383dda94895 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3781302d19a4a8a9342e98ea95ddc93" member_type: NON_VOTER last_known_addr { host: "127.27.180.189" port: 45319 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:55:30.269164 8659 master.cc:537] Master@127.27.180.189:45319 shutting down...
I20260811 18:55:30.276068 8859 sys_catalog.cc:455] T 00000000000000000000000000000000 P fec531dd027244268997c383dda94895 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "fec531dd027244268997c383dda94895" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "fec531dd027244268997c383dda94895" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 46847 } } peers { permanent_uuid: "e3781302d19a4a8a9342e98ea95ddc93" member_type: NON_VOTER last_known_addr { host: "127.27.180.189" port: 45319 } attrs { promote: true } } }
I20260811 18:55:30.276278 8859 sys_catalog.cc:458] T 00000000000000000000000000000000 P fec531dd027244268997c383dda94895 [sys.catalog]: This master's current role is: LEADER
I20260811 18:55:30.273061 8743 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e3781302d19a4a8a9342e98ea95ddc93 [term 1 LEARNER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER e3781302d19a4a8a9342e98ea95ddc93 (127.27.180.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "fec531dd027244268997c383dda94895" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 46847 } } peers { permanent_uuid: "e3781302d19a4a8a9342e98ea95ddc93" member_type: NON_VOTER last_known_addr { host: "127.27.180.189" port: 45319 } attrs { promote: true } } }
I20260811 18:55:30.278650 8812 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3781302d19a4a8a9342e98ea95ddc93 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fec531dd027244268997c383dda94895. Latest consensus state: current_term: 1 leader_uuid: "fec531dd027244268997c383dda94895" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "fec531dd027244268997c383dda94895" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 46847 } } peers { permanent_uuid: "e3781302d19a4a8a9342e98ea95ddc93" member_type: NON_VOTER last_known_addr { host: "127.27.180.189" port: 45319 } attrs { promote: true } } }
I20260811 18:55:30.278759 8812 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3781302d19a4a8a9342e98ea95ddc93 [sys.catalog]: This master's current role is: LEARNER
I20260811 18:55:30.278846 8812 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3781302d19a4a8a9342e98ea95ddc93 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "fec531dd027244268997c383dda94895" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "fec531dd027244268997c383dda94895" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 46847 } } peers { permanent_uuid: "e3781302d19a4a8a9342e98ea95ddc93" member_type: NON_VOTER last_known_addr { host: "127.27.180.189" port: 45319 } attrs { promote: true } } }
I20260811 18:55:30.278916 8812 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3781302d19a4a8a9342e98ea95ddc93 [sys.catalog]: This master's current role is: LEARNER
I20260811 18:55:30.278971 8812 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3781302d19a4a8a9342e98ea95ddc93 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "fec531dd027244268997c383dda94895" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "fec531dd027244268997c383dda94895" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 46847 } } peers { permanent_uuid: "e3781302d19a4a8a9342e98ea95ddc93" member_type: NON_VOTER last_known_addr { host: "127.27.180.189" port: 45319 } attrs { promote: true } } }
I20260811 18:55:30.279021 8812 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3781302d19a4a8a9342e98ea95ddc93 [sys.catalog]: This master's current role is: LEARNER
I20260811 18:55:30.282661 8863 mvcc.cc:205] Tried to move back new op lower bound from 7317399672069021696 to 7317399671949107200. Current Snapshot: MvccSnapshot[applied={T|T < 7317399671984803840}]
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.27.180.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/krb5kdc/oryx_127.27.180.188.keytab.
Entry for principal oryx/127.27.180.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/krb5kdc/oryx_127.27.180.188.keytab.
I20260811 18:55:30.286417 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.188:34747
--webserver_interface=127.27.180.188
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/krb5kdc/oryx_127.27.180.188.keytab
--principal=oryx/127.27.180.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:33071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.180.190:46847,127.27.180.189:45319,127.27.180.188:34747
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 18:55:30.288198 8659 raft_consensus.cc:2229] T 00000000000000000000000000000000 P e3781302d19a4a8a9342e98ea95ddc93 [term 1 LEARNER]: Raft consensus shutting down.
I20260811 18:55:30.288523 8659 raft_consensus.cc:2258] T 00000000000000000000000000000000 P e3781302d19a4a8a9342e98ea95ddc93 [term 1 LEARNER]: Raft consensus is shut down!
I20260811 18:55:30.288616 8659 tablet_replica.cc:330] T 00000000000000000000000000000000 P e3781302d19a4a8a9342e98ea95ddc93: stopping tablet replica
I20260811 18:55:30.294958 8659 master.cc:552] Master@127.27.180.189:45319 shutdown complete.
I20260811 18:55:30.295055 8659 master_runner.cc:305] Clearing existing system tablet
I20260811 18:55:30.295360 8659 fs_manager.cc:359] Metadata directory not provided
I20260811 18:55:30.295466 8659 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:55:30.301461 8659 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.001s sys 0.000s
I20260811 18:55:30.302434 8879 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:30.302639 8659 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:55:30.302752 8659 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 18:55:30.302791 8659 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/master-1/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/master-1/wal
uuid: "e3781302d19a4a8a9342e98ea95ddc93"
format_stamp: "Formatted at 2026-08-11 18:55:30 on dist-test-slave-2j7r"
server_key: "959ee01066efa93df4ad009cb06d50f1"
server_key_iv: "36c6f64bd507ef311d374d291468b95c"
server_key_version: "encryptionkey@0"
I20260811 18:55:30.304823 8659 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P e3781302d19a4a8a9342e98ea95ddc93: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:55:30.307531 8659 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P e3781302d19a4a8a9342e98ea95ddc93: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260811 18:55:30.307634 8659 log.cc:1196] T 00000000000000000000000000000000 P e3781302d19a4a8a9342e98ea95ddc93: Deleting WAL directory at /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260811 18:55:30.307965 8659 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P e3781302d19a4a8a9342e98ea95ddc93: Deleting consensus metadata
I20260811 18:55:30.308135 8659 master_runner.cc:315] Copying system tablet from 127.27.180.190:46847
I20260811 18:55:30.309779 8659 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P e3781302d19a4a8a9342e98ea95ddc93: tablet copy: Beginning tablet copy session from remote peer at address 127.27.180.190:46847
I20260811 18:55:30.326086 7828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:30.316113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36118 (local address 127.27.180.190:46847)
0811 18:55:30.316271 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:30.316276 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:30.317119 (+ 843us) server_negotiation.cc:407] Connection header received
0811 18:55:30.317403 (+ 284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:30.317407 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:30.317467 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:30.317525 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:30.318206 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:30.319058 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:30.320278 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:30.320639 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:30.320715 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:30.322227 (+ 1512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:30.322232 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:30.322235 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:30.322266 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:30.324099 (+ 1833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:30.324643 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:30.324647 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:30.324649 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:30.324739 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:30.325063 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:30.325067 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:30.325069 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:30.325311 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:30.325386 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:30.325749 (+ 363us) server_negotiation.cc:299] Negotiation successful
0811 18:55:30.325888 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 18:55:30.326640 8889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:30.316482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:46847 (local address 127.0.0.1:36118)
0811 18:55:30.316981 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:30.316996 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:30.317267 (+ 271us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:30.317691 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:30.317694 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:30.317944 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:30.318063 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:30.318070 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:30.319211 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:30.319214 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:30.320129 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:30.320142 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:30.320904 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:30.320908 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:30.321008 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:30.321594 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:30.321610 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:30.322056 (+ 446us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:55:30.324251 (+ 2195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:30.324254 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:30.324256 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:30.324521 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:30.324849 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:30.324853 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:30.324855 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:30.324963 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:30.325462 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:55:30.325465 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:55:30.325568 (+ 103us) client_negotiation.cc:769] Sending connection context
0811 18:55:30.326306 (+ 738us) client_negotiation.cc:240] Negotiation successful
0811 18:55:30.326460 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":232,"threads_started":1}
I20260811 18:55:30.327289 7801 tablet_copy_service.cc:140] P fec531dd027244268997c383dda94895: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer e3781302d19a4a8a9342e98ea95ddc93 ({username='oryx', principal='oryx/127.27.180.189@KRBTEST.COM'} at 127.0.0.1:36118)
I20260811 18:55:30.327494 7801 tablet_copy_service.cc:161] P fec531dd027244268997c383dda94895: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer e3781302d19a4a8a9342e98ea95ddc93 at {username='oryx', principal='oryx/127.27.180.189@KRBTEST.COM'} at 127.0.0.1:36118: session id = e3781302d19a4a8a9342e98ea95ddc93-00000000000000000000000000000000
I20260811 18:55:30.329384 7801 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P fec531dd027244268997c383dda94895: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 18:55:30.331223 8659 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260811 18:55:30.334007 8659 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P e3781302d19a4a8a9342e98ea95ddc93: tablet copy: Starting download of 0 data blocks...
I20260811 18:55:30.334544 8659 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P e3781302d19a4a8a9342e98ea95ddc93: tablet copy: Starting download of 1 WAL segments...
I20260811 18:55:30.337524 8659 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P e3781302d19a4a8a9342e98ea95ddc93: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 18:55:30.339320 7801 tablet_copy_service.cc:342] P fec531dd027244268997c383dda94895: Request end of tablet copy session e3781302d19a4a8a9342e98ea95ddc93-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.27.180.189@KRBTEST.COM'} at 127.0.0.1:36118
I20260811 18:55:30.339448 7801 tablet_copy_service.cc:433] P fec531dd027244268997c383dda94895: ending tablet copy session e3781302d19a4a8a9342e98ea95ddc93-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer e3781302d19a4a8a9342e98ea95ddc93
I20260811 18:55:30.355516 8659 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:55:30.357339 8890 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:30.362083 8892 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:55:30.362317 8659 server_base.cc:1034] running on GCE node
W20260811 18:55:30.362533 8897 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:55:30.362774 8659 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:55:30.363092 8659 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:55:30.366315 8659 hybrid_clock.cc:648] HybridClock initialized: now 1786474530366302 us; error 30 us; skew 500 ppm
Aug 11 18:55:30 dist-test-slave-2j7r krb5kdc[7666](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474530, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.180.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:55:30.368028 8659 init.cc:377] Logged in from keytab as oryx/127.27.180.189@KRBTEST.COM (short username oryx)
I20260811 18:55:30.368525 8659 webserver.cc:460] Webserver started at http://127.27.180.189:44975/ using document root <none> and password file <none>
I20260811 18:55:30.368705 8659 fs_manager.cc:359] Metadata directory not provided
I20260811 18:55:30.368741 8659 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:55:30.370186 8659 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 18:55:30.371266 8908 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:30.371532 8659 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 18:55:30.371595 8659 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/master-1/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/master-1/wal
uuid: "e3781302d19a4a8a9342e98ea95ddc93"
format_stamp: "Formatted at 2026-08-11 18:55:30 on dist-test-slave-2j7r"
server_key: "959ee01066efa93df4ad009cb06d50f1"
server_key_iv: "36c6f64bd507ef311d374d291468b95c"
server_key_version: "encryptionkey@0"
I20260811 18:55:30.371668 8659 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:55:30.380973 8659 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:55:30.383045 8659 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:55:30.387822 8659 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.189:45319
I20260811 18:55:30.388451 8659 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/master-1/data/info.pb
I20260811 18:55:30.390010 9013 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.189:45319 every 8 connection(s)
I20260811 18:55:30.391814 9015 sys_catalog.cc:263] Verifying existing consensus state
I20260811 18:55:30.393030 9015 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3781302d19a4a8a9342e98ea95ddc93: Bootstrap starting.
I20260811 18:55:30.400836 9015 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3781302d19a4a8a9342e98ea95ddc93: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 18:55:30.401233 9015 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3781302d19a4a8a9342e98ea95ddc93: Bootstrap complete.
I20260811 18:55:30.401871 9015 raft_consensus.cc:348] T 00000000000000000000000000000000 P e3781302d19a4a8a9342e98ea95ddc93 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "fec531dd027244268997c383dda94895" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 46847 } } peers { permanent_uuid: "e3781302d19a4a8a9342e98ea95ddc93" member_type: NON_VOTER last_known_addr { host: "127.27.180.189" port: 45319 } attrs { promote: true } }
I20260811 18:55:30.402132 9015 raft_consensus.cc:729] T 00000000000000000000000000000000 P e3781302d19a4a8a9342e98ea95ddc93 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e3781302d19a4a8a9342e98ea95ddc93, State: Initialized, Role: LEARNER
I20260811 18:55:30.402302 9015 consensus_queue.cc:260] T 00000000000000000000000000000000 P e3781302d19a4a8a9342e98ea95ddc93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "fec531dd027244268997c383dda94895" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 46847 } } peers { permanent_uuid: "e3781302d19a4a8a9342e98ea95ddc93" member_type: NON_VOTER last_known_addr { host: "127.27.180.189" port: 45319 } attrs { promote: true } }
I20260811 18:55:30.402871 9015 sys_catalog.cc:564] T 00000000000000000000000000000000 P e3781302d19a4a8a9342e98ea95ddc93 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:55:30.406911 9043 catalog_manager.cc:1259] Loaded cluster ID: c733e261438841f4aa5d657b05032fa9
I20260811 18:55:30.407071 9043 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e3781302d19a4a8a9342e98ea95ddc93: loading cluster ID for follower catalog manager: success
I20260811 18:55:30.409444 9043 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e3781302d19a4a8a9342e98ea95ddc93: acquiring CA information for follower catalog manager: success
I20260811 18:55:30.410959 9043 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e3781302d19a4a8a9342e98ea95ddc93: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 18:55:30.411216 9020 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3781302d19a4a8a9342e98ea95ddc93 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "fec531dd027244268997c383dda94895" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 46847 } } peers { permanent_uuid: "e3781302d19a4a8a9342e98ea95ddc93" member_type: NON_VOTER last_known_addr { host: "127.27.180.189" port: 45319 } attrs { promote: true } } }
I20260811 18:55:30.411387 9020 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3781302d19a4a8a9342e98ea95ddc93 [sys.catalog]: This master's current role is: LEARNER
W20260811 18:55:30.452647 8875 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:55:30.452996 8875 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:55:30.453058 8875 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:55:30.459178 8875 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:55:30.459259 8875 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:55:30.459295 8875 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:55:30.459331 8875 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:55:30.459360 8875 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:55:30.467276 8875 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:33071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.27.180.190:46847,127.27.180.189:45319,127.27.180.188:34747
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/krb5kdc/oryx_127.27.180.188.keytab
--principal=oryx/127.27.180.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.188:34747
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.27.180.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8875
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:55:30.468550 8875 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:55:30.469772 8875 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:55:30.478185 9055 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:55:30.478448 8875 server_base.cc:1034] running on GCE node
W20260811 18:55:30.478174 9050 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:30.478200 9052 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:55:30.479198 8875 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:55:30.480273 8875 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:55:30.481431 8875 hybrid_clock.cc:648] HybridClock initialized: now 1786474530481403 us; error 46 us; skew 500 ppm
Aug 11 18:55:30 dist-test-slave-2j7r krb5kdc[7666](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474530, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.180.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:55:30.484229 8875 init.cc:377] Logged in from keytab as oryx/127.27.180.188@KRBTEST.COM (short username oryx)
I20260811 18:55:30.485229 8875 webserver.cc:460] Webserver started at http://127.27.180.188:38417/ using document root <none> and password file <none>
I20260811 18:55:30.485843 8875 fs_manager.cc:359] Metadata directory not provided
I20260811 18:55:30.485944 8875 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:55:30.486167 8875 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:55:30.488297 8875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/master-2/data/instance:
uuid: "9149de133657420baaadd406d6f504fd"
format_stamp: "Formatted at 2026-08-11 18:55:30 on dist-test-slave-2j7r"
server_key: "3675630bcba495c3a5fda1a4cb1bf68a"
server_key_iv: "71b0153d6d6d923206fe7a3509467ec3"
server_key_version: "encryptionkey@0"
I20260811 18:55:30.488898 8875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/master-2/wal/instance:
uuid: "9149de133657420baaadd406d6f504fd"
format_stamp: "Formatted at 2026-08-11 18:55:30 on dist-test-slave-2j7r"
server_key: "3675630bcba495c3a5fda1a4cb1bf68a"
server_key_iv: "71b0153d6d6d923206fe7a3509467ec3"
server_key_version: "encryptionkey@0"
I20260811 18:55:30.493156 8875 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260811 18:55:30.495568 9068 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:30.496887 8875 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260811 18:55:30.497002 8875 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/master-2/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/master-2/wal
uuid: "9149de133657420baaadd406d6f504fd"
format_stamp: "Formatted at 2026-08-11 18:55:30 on dist-test-slave-2j7r"
server_key: "3675630bcba495c3a5fda1a4cb1bf68a"
server_key_iv: "71b0153d6d6d923206fe7a3509467ec3"
server_key_version: "encryptionkey@0"
I20260811 18:55:30.497115 8875 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:55:30.530417 8875 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:55:30.533317 8875 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:55:30.533475 8875 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:55:30.540892 8109 heartbeater.cc:503] Master 127.27.180.190:46847 was elected leader, sending a full tablet report...
I20260811 18:55:30.543205 8875 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.188:34747
I20260811 18:55:30.543438 9169 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.188:34747 every 8 connection(s)
I20260811 18:55:30.544435 8875 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/master-2/data/info.pb
I20260811 18:55:30.547181 9172 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:55:30.549710 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 8875
I20260811 18:55:30.551191 9172 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.180.190" port: 46847 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 18:55:30 dist-test-slave-2j7r krb5kdc[7666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474530, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.180.188@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM
Aug 11 18:55:30 dist-test-slave-2j7r krb5kdc[7666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474529, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.27.180.188@KRBTEST.COM
I20260811 18:55:30.569293 7828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:30.552902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36134 (local address 127.27.180.190:46847)
0811 18:55:30.553014 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:30.553018 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:30.553401 (+ 383us) server_negotiation.cc:407] Connection header received
0811 18:55:30.553595 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:30.553599 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:30.553652 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:30.553718 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:30.554674 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:30.555325 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:30.556314 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:30.556531 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:30.556625 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:30.565594 (+ 8969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:30.565599 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:30.565601 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:30.565632 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:30.567081 (+ 1449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:30.567610 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:30.567613 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:30.567615 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:30.567694 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:30.568003 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:30.568006 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:30.568008 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:30.568229 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:30.568350 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:30.568957 (+ 607us) server_negotiation.cc:299] Negotiation successful
0811 18:55:30.569080 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 18:55:30.570353 9176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:30.552796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:46847 (local address 127.0.0.1:36134)
0811 18:55:30.553271 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:30.553320 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:30.553493 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:30.553775 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:30.553783 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:30.554202 (+ 419us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:30.554511 (+ 309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:30.554524 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:30.555469 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:30.555473 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:30.556196 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:30.556205 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:30.556698 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:30.556703 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:30.556985 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:30.563488 (+ 6503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:30.563509 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:30.565436 (+ 1927us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:55:30.567208 (+ 1772us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:30.567218 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:30.567227 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:30.567510 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:30.567778 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:30.567781 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:30.567783 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:30.567914 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:30.568342 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:55:30.568348 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:55:30.568690 (+ 342us) client_negotiation.cc:769] Sending connection context
0811 18:55:30.568897 (+ 207us) client_negotiation.cc:240] Negotiation successful
0811 18:55:30.569246 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"mutex_wait_us":5,"thread_start_us":88,"threads_started":1}
I20260811 18:55:30.570353 9175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:30.551228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54672 (local address 127.27.180.188:34747)
0811 18:55:30.551798 (+ 570us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:30.551822 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:30.551874 (+ 52us) server_negotiation.cc:407] Connection header received
0811 18:55:30.552552 (+ 678us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:30.552569 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:30.552992 (+ 423us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:30.553888 (+ 896us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:30.554377 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:30.555324 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:30.556577 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:30.557040 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:30.557185 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:30.566208 (+ 9023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:30.566215 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:30.566229 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:30.566256 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:30.568035 (+ 1779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:30.568533 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:30.568539 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:30.568541 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:30.568616 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:30.568863 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:30.568867 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:30.568869 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:30.569124 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:30.569913 (+ 789us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:30.570021 (+ 108us) server_negotiation.cc:299] Negotiation successful
0811 18:55:30.570159 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":64,"server-negotiator.queue_time_us":395,"thread_start_us":124,"threads_started":1}
I20260811 18:55:30.572472 9172 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.180.189" port: 45319 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 18:55:30 dist-test-slave-2j7r krb5kdc[7666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474530, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.180.188@KRBTEST.COM for oryx/127.27.180.189@KRBTEST.COM
I20260811 18:55:30.580017 9176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:30.572965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.189:45319 (local address 127.0.0.1:44438)
0811 18:55:30.573127 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:30.573139 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:30.573204 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:30.573584 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:30.573586 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:30.573736 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:30.573822 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:30.573828 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:30.574692 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:30.574695 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:30.575238 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:30.575245 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:30.575660 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:30.575663 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:30.575775 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:30.576169 (+ 394us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:30.576179 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:30.577263 (+ 1084us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:55:30.578748 (+ 1485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:30.578751 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:30.578753 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:30.578926 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:30.579168 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:30.579171 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:30.579173 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:30.579249 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:30.579578 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:55:30.579581 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:55:30.579665 (+ 84us) client_negotiation.cc:769] Sending connection context
0811 18:55:30.579754 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 18:55:30.579871 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":86}
I20260811 18:55:30.580080 9179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:30.573074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44438 (local address 127.27.180.189:45319)
0811 18:55:30.573394 (+ 320us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:30.573397 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:30.573409 (+ 12us) server_negotiation.cc:407] Connection header received
0811 18:55:30.573443 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:30.573445 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:30.573481 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:30.573555 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:30.573929 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:30.574592 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:30.575338 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:30.575568 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:30.575652 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:30.577394 (+ 1742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:30.577396 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:30.577398 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:30.577421 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:30.578645 (+ 1224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:30.579010 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:30.579013 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:30.579014 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:30.579088 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:30.579326 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:30.579329 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:30.579331 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:30.579491 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:30.579574 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:30.579815 (+ 241us) server_negotiation.cc:299] Negotiation successful
0811 18:55:30.579925 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":227,"thread_start_us":122,"threads_started":1}
I20260811 18:55:30.580724 9172 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.180.188" port: 34747 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 18:55:30 dist-test-slave-2j7r krb5kdc[7666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474530, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.180.188@KRBTEST.COM for oryx/127.27.180.188@KRBTEST.COM
I20260811 18:55:30.588387 9176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:30.581250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.188:34747 (local address 127.0.0.1:54676)
0811 18:55:30.581350 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:30.581363 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:30.581435 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:30.581657 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:30.581659 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:30.581777 (+ 118us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:30.581844 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:30.581848 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:30.582792 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:30.582794 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:30.583273 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:30.583279 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:30.583815 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:30.583817 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:30.583890 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:30.584275 (+ 385us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:30.584285 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:30.585449 (+ 1164us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:55:30.587135 (+ 1686us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:30.587138 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:30.587140 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:30.587315 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:30.587547 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:30.587550 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:30.587552 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:30.587625 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:30.587939 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:55:30.587942 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:55:30.588020 (+ 78us) client_negotiation.cc:769] Sending connection context
0811 18:55:30.588100 (+ 80us) client_negotiation.cc:240] Negotiation successful
0811 18:55:30.588220 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":3}
I20260811 18:55:30.588388 9175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:30.581372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54676 (local address 127.27.180.188:34747)
0811 18:55:30.581477 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:30.581481 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:30.581493 (+ 12us) server_negotiation.cc:407] Connection header received
0811 18:55:30.581525 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:30.581529 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:30.581568 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:30.581647 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:30.581929 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:30.582681 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:30.583361 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:30.583720 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:30.583856 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:30.585579 (+ 1723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:30.585583 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:30.585586 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:30.585613 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:30.587043 (+ 1430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:30.587397 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:30.587401 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:30.587402 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:30.587473 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:30.587697 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:30.587700 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:30.587702 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:30.587852 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:30.587940 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:30.588152 (+ 212us) server_negotiation.cc:299] Negotiation successful
0811 18:55:30.588267 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":52,"server-negotiator.queue_time_us":37}
I20260811 18:55:30.592336 9172 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9149de133657420baaadd406d6f504fd: Bootstrap starting.
I20260811 18:55:30.595448 9172 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9149de133657420baaadd406d6f504fd: Neither blocks nor log segments found. Creating new log.
I20260811 18:55:30.596380 9172 log.cc:824] T 00000000000000000000000000000000 P 9149de133657420baaadd406d6f504fd: Log is configured to *not* fsync() on all Append() calls
I20260811 18:55:30.598619 9172 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9149de133657420baaadd406d6f504fd: No bootstrap required, opened a new log
I20260811 18:55:30.601703 9172 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9149de133657420baaadd406d6f504fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fec531dd027244268997c383dda94895" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 46847 } } peers { permanent_uuid: "e3781302d19a4a8a9342e98ea95ddc93" member_type: VOTER last_known_addr { host: "127.27.180.189" port: 45319 } } peers { permanent_uuid: "9149de133657420baaadd406d6f504fd" member_type: VOTER last_known_addr { host: "127.27.180.188" port: 34747 } }
I20260811 18:55:30.601913 9172 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9149de133657420baaadd406d6f504fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:55:30.601956 9172 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9149de133657420baaadd406d6f504fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9149de133657420baaadd406d6f504fd, State: Initialized, Role: FOLLOWER
I20260811 18:55:30.602340 9172 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9149de133657420baaadd406d6f504fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fec531dd027244268997c383dda94895" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 46847 } } peers { permanent_uuid: "e3781302d19a4a8a9342e98ea95ddc93" member_type: VOTER last_known_addr { host: "127.27.180.189" port: 45319 } } peers { permanent_uuid: "9149de133657420baaadd406d6f504fd" member_type: VOTER last_known_addr { host: "127.27.180.188" port: 34747 } }
I20260811 18:55:30.602752 9183 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9149de133657420baaadd406d6f504fd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fec531dd027244268997c383dda94895" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 46847 } } peers { permanent_uuid: "e3781302d19a4a8a9342e98ea95ddc93" member_type: VOTER last_known_addr { host: "127.27.180.189" port: 45319 } } peers { permanent_uuid: "9149de133657420baaadd406d6f504fd" member_type: VOTER last_known_addr { host: "127.27.180.188" port: 34747 } } }
I20260811 18:55:30.602864 9183 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9149de133657420baaadd406d6f504fd [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:55:30.603499 9172 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9149de133657420baaadd406d6f504fd [sys.catalog]: configured and running, proceeding with master startup.
W20260811 18:55:30.607865 9198 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9149de133657420baaadd406d6f504fd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:55:30.607957 9198 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:55:30.610935 8875 master_runner.cc:428] Detected that this master 9149de133657420baaadd406d6f504fd is joining an existing cluster
I20260811 18:55:30.610991 8875 master_runner.cc:432] Initiating AddMaster RPC to add 127.27.180.188:34747
I20260811 18:55:30.621402 9214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:30.613039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.188:34747 (local address 127.0.0.1:54692)
0811 18:55:30.614147 (+ 1108us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:30.614162 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:30.614265 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:30.614487 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:30.614491 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:30.614742 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:30.614895 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:30.614913 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:30.615910 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:30.615914 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:30.616818 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:30.616830 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:30.617423 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:30.617427 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:30.617522 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:30.618010 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:30.618025 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:30.618422 (+ 397us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:55:30.619970 (+ 1548us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:30.619973 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:30.619975 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:30.620215 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:30.620437 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:30.620440 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:30.620442 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:30.620528 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:30.620836 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:55:30.620839 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:55:30.620936 (+ 97us) client_negotiation.cc:769] Sending connection context
0811 18:55:30.621030 (+ 94us) client_negotiation.cc:240] Negotiation successful
0811 18:55:30.621186 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1046,"mutex_wait_us":23,"thread_start_us":572,"threads_started":1}
I20260811 18:55:30.621416 9175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:30.613516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54692 (local address 127.27.180.188:34747)
0811 18:55:30.613622 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:30.613625 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:30.614242 (+ 617us) server_negotiation.cc:407] Connection header received
0811 18:55:30.614336 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:30.614339 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:30.614379 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:30.614460 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:30.615008 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:30.615790 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:30.616941 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:30.617305 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:30.617400 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:30.618567 (+ 1167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:30.618570 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:30.618572 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:30.618595 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:30.619870 (+ 1275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:30.620295 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:30.620298 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:30.620299 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:30.620366 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:30.620602 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:30.620604 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:30.620605 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:30.620764 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:30.620832 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:30.621074 (+ 242us) server_negotiation.cc:299] Negotiation successful
0811 18:55:30.621254 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":38,"server-negotiator.queue_time_us":43}
I20260811 18:55:30.623867 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 7873
I20260811 18:55:30.639847 7828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:30.613232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36138 (local address 127.27.180.190:46847)
0811 18:55:30.613420 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:30.613425 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:30.621358 (+ 7933us) server_negotiation.cc:407] Connection header received
0811 18:55:30.621534 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:30.621537 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:30.621586 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:30.621633 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:30.622177 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:30.622726 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:30.623652 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:30.623945 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:30.624021 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:30.636452 (+ 12431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:30.636456 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:30.636459 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:30.636502 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:30.638069 (+ 1567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:30.638565 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:30.638569 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:30.638570 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:30.638660 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:30.638948 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:30.638951 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:30.638952 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:30.639157 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:30.639223 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:30.639551 (+ 328us) server_negotiation.cc:299] Negotiation successful
0811 18:55:30.639687 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260811 18:55:30.640354 9212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:30.613811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:46847 (local address 127.0.0.1:36138)
0811 18:55:30.621272 (+ 7461us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:30.621285 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:30.621419 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:30.621768 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:30.621771 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:30.621970 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:30.622068 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:30.622074 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:30.622829 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:30.622832 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:30.623522 (+ 690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:30.623531 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:30.624183 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:30.624186 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:30.624282 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:30.635749 (+ 11467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:30.635767 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:30.636264 (+ 497us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:55:30.638198 (+ 1934us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:30.638202 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:30.638204 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:30.638447 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:30.638776 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:30.638778 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:30.638780 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:30.638860 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:30.639292 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:55:30.639295 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:55:30.639382 (+ 87us) client_negotiation.cc:769] Sending connection context
0811 18:55:30.640036 (+ 654us) client_negotiation.cc:240] Negotiation successful
0811 18:55:30.640182 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7392,"thread_start_us":109,"threads_started":1}
I20260811 18:55:30.642392 9179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:30.613416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44444 (local address 127.27.180.189:45319)
0811 18:55:30.613541 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:30.613546 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:30.613967 (+ 421us) server_negotiation.cc:407] Connection header received
0811 18:55:30.614137 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:30.614141 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:30.614201 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:30.614285 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:30.614975 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:30.615807 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:30.616874 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:30.617279 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:30.617390 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:30.638419 (+ 21029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:30.638423 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:30.638426 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:30.638457 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:30.640061 (+ 1604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:30.641048 (+ 987us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:30.641052 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:30.641054 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:30.641172 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:30.641468 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:30.641472 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:30.641474 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:30.641683 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:30.641751 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:30.642109 (+ 358us) server_negotiation.cc:299] Negotiation successful
0811 18:55:30.642238 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 18:55:30.643791 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.180.129:32985
--local_ip_for_outbound_sockets=127.27.180.129
--tserver_master_addrs=127.27.180.190:46847
--webserver_port=35595
--webserver_interface=127.27.180.129
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/krb5kdc/oryx_127.27.180.129.keytab
--principal=oryx/127.27.180.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:33071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.27.180.190:46847,127.27.180.189:45319
--tserver_master_addrs=127.27.180.190:46847,127.27.180.189:45319,127.27.180.188:34747 with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 18:55:30.643996 9213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:30.612999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.189:45319 (local address 127.0.0.1:44444)
0811 18:55:30.613880 (+ 881us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:30.613892 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:30.614031 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:30.614424 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:30.614428 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:30.614714 (+ 286us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:30.614854 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:30.614861 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:30.615991 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:30.615994 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:30.616751 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:30.616761 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:30.617516 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:30.617521 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:30.617632 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:30.637829 (+ 20197us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:30.637843 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:30.638235 (+ 392us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:55:30.640682 (+ 2447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:30.640685 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:30.640687 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:30.640932 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:30.641284 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:30.641287 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:30.641288 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:30.641370 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:30.641819 (+ 449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:55:30.641822 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:55:30.641921 (+ 99us) client_negotiation.cc:769] Sending connection context
0811 18:55:30.643663 (+ 1742us) client_negotiation.cc:240] Negotiation successful
0811 18:55:30.643807 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":785,"mutex_wait_us":38,"thread_start_us":190,"threads_started":1}
Aug 11 18:55:30 dist-test-slave-2j7r krb5kdc[7666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474529, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.180.190@KRBTEST.COM for oryx/127.27.180.188@KRBTEST.COM
I20260811 18:55:30.653851 9175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:30.645165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54694 (local address 127.27.180.188:34747)
0811 18:55:30.645265 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:30.645269 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:30.645487 (+ 218us) server_negotiation.cc:407] Connection header received
0811 18:55:30.645610 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:30.645613 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:30.645665 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:30.645737 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:30.646209 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:30.646916 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:30.647994 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:30.648314 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:30.648412 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:30.650655 (+ 2243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:30.650660 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:30.650662 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:30.650688 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:30.652193 (+ 1505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:30.652659 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:30.652663 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:30.652665 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:30.652743 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:30.653019 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:30.653022 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:30.653024 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:30.653236 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:30.653327 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:30.653605 (+ 278us) server_negotiation.cc:299] Negotiation successful
0811 18:55:30.653710 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 18:55:30.653870 8846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:30.644911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.188:34747 (local address 127.0.0.1:54694)
0811 18:55:30.645419 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:30.645430 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:30.645510 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:30.645785 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:30.645788 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:30.645993 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:30.646097 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:30.646104 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:30.647019 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:30.647023 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:30.647861 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:30.647871 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:30.648427 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:30.648431 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:30.648525 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:30.648975 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:30.648989 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:30.650516 (+ 1527us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:55:30.652307 (+ 1791us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:30.652310 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:30.652313 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:30.652554 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:30.652829 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:30.652833 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:30.652834 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:30.652927 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:30.653331 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:55:30.653334 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:55:30.653440 (+ 106us) client_negotiation.cc:769] Sending connection context
0811 18:55:30.653553 (+ 113us) client_negotiation.cc:240] Negotiation successful
0811 18:55:30.653694 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":443}
I20260811 18:55:30.655233 7768 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.27.180.188:34747: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "9149de133657420baaadd406d6f504fd"
member_type: NON_VOTER
last_known_addr {
host: "127.27.180.188"
port: 34747
}
attrs {
promote: true
}
}
dest_uuid: "fec531dd027244268997c383dda94895"
cas_config_opid_index: 5
I20260811 18:55:30.655860 7768 consensus_queue.cc:237] T 00000000000000000000000000000000 P fec531dd027244268997c383dda94895 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "fec531dd027244268997c383dda94895" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 46847 } } peers { permanent_uuid: "e3781302d19a4a8a9342e98ea95ddc93" member_type: NON_VOTER last_known_addr { host: "127.27.180.189" port: 45319 } attrs { promote: true } } peers { permanent_uuid: "9149de133657420baaadd406d6f504fd" member_type: NON_VOTER last_known_addr { host: "127.27.180.188" port: 34747 } attrs { promote: true } }
I20260811 18:55:30.656888 8851 raft_consensus.cc:2932] T 00000000000000000000000000000000 P fec531dd027244268997c383dda94895 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 9149de133657420baaadd406d6f504fd (127.27.180.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "fec531dd027244268997c383dda94895" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 46847 } } peers { permanent_uuid: "e3781302d19a4a8a9342e98ea95ddc93" member_type: NON_VOTER last_known_addr { host: "127.27.180.189" port: 45319 } attrs { promote: true } } peers { permanent_uuid: "9149de133657420baaadd406d6f504fd" member_type: NON_VOTER last_known_addr { host: "127.27.180.188" port: 34747 } attrs { promote: true } } }
I20260811 18:55:30.657738 9112 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9149de133657420baaadd406d6f504fd [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:55:30.658376 8851 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.27.180.188:34747
I20260811 18:55:30.658567 8851 sys_catalog.cc:455] T 00000000000000000000000000000000 P fec531dd027244268997c383dda94895 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "fec531dd027244268997c383dda94895" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "fec531dd027244268997c383dda94895" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 46847 } } peers { permanent_uuid: "e3781302d19a4a8a9342e98ea95ddc93" member_type: NON_VOTER last_known_addr { host: "127.27.180.189" port: 45319 } attrs { promote: true } } peers { permanent_uuid: "9149de133657420baaadd406d6f504fd" member_type: NON_VOTER last_known_addr { host: "127.27.180.188" port: 34747 } attrs { promote: true } } }
I20260811 18:55:30.658656 8851 sys_catalog.cc:458] T 00000000000000000000000000000000 P fec531dd027244268997c383dda94895 [sys.catalog]: This master's current role is: LEADER
I20260811 18:55:30.659189 8875 master.cc:537] Master@127.27.180.188:34747 shutting down...
I20260811 18:55:30.659205 9112 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9149de133657420baaadd406d6f504fd [term 1 FOLLOWER]: Refusing update from remote peer fec531dd027244268997c383dda94895: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260811 18:55:30.659936 8859 consensus_queue.cc:1035] T 00000000000000000000000000000000 P fec531dd027244268997c383dda94895 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9149de133657420baaadd406d6f504fd" member_type: NON_VOTER last_known_addr { host: "127.27.180.188" port: 34747 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 18:55:30.661444 7739 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260811 18:55:30.663512 8846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:30.657055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.189:45319 (local address 127.0.0.1:44448)
0811 18:55:30.657307 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:30.657319 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:30.657381 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:30.659295 (+ 1914us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:30.659298 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:30.659314 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:55:30.659718 (+ 404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:30.659726 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:30.660696 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:30.660699 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:30.662222 (+ 1523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:30.662232 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:30.663125 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:30.663129 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:30.663217 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:30.663264 (+ 47us) client_negotiation.cc:769] Sending connection context
0811 18:55:30.663332 (+ 68us) client_negotiation.cc:240] Negotiation successful
0811 18:55:30.663399 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187}
I20260811 18:55:30.663544 9179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:30.658819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44448 (local address 127.27.180.189:45319)
0811 18:55:30.658947 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:30.658951 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:30.658987 (+ 36us) server_negotiation.cc:407] Connection header received
0811 18:55:30.659031 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:30.659033 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:30.659091 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:30.659179 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:55:30.659858 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:30.660559 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:30.662354 (+ 1795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:30.663015 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:30.663135 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:30.663268 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:30.663357 (+ 89us) server_negotiation.cc:299] Negotiation successful
0811 18:55:30.663426 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 18:55:30.664345 8956 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e3781302d19a4a8a9342e98ea95ddc93 [term 1 LEARNER]: Refusing update from remote peer fec531dd027244268997c383dda94895: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260811 18:55:30.664870 8859 consensus_queue.cc:1035] T 00000000000000000000000000000000 P fec531dd027244268997c383dda94895 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3781302d19a4a8a9342e98ea95ddc93" member_type: NON_VOTER last_known_addr { host: "127.27.180.189" port: 45319 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260811 18:55:30.665812 7739 consensus_peers.cc:487] T 00000000000000000000000000000000 P fec531dd027244268997c383dda94895 -> Peer 9149de133657420baaadd406d6f504fd (127.27.180.188:34747): Couldn't send request to peer 9149de133657420baaadd406d6f504fd. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260811 18:55:30.665871 8956 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e3781302d19a4a8a9342e98ea95ddc93 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 9149de133657420baaadd406d6f504fd (127.27.180.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "fec531dd027244268997c383dda94895" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 46847 } } peers { permanent_uuid: "e3781302d19a4a8a9342e98ea95ddc93" member_type: NON_VOTER last_known_addr { host: "127.27.180.189" port: 45319 } attrs { promote: true } } peers { permanent_uuid: "9149de133657420baaadd406d6f504fd" member_type: NON_VOTER last_known_addr { host: "127.27.180.188" port: 34747 } attrs { promote: true } } }
I20260811 18:55:30.667438 8875 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 9149de133657420baaadd406d6f504fd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:55:30.667465 9020 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3781302d19a4a8a9342e98ea95ddc93 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fec531dd027244268997c383dda94895. Latest consensus state: current_term: 1 leader_uuid: "fec531dd027244268997c383dda94895" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "fec531dd027244268997c383dda94895" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 46847 } } peers { permanent_uuid: "e3781302d19a4a8a9342e98ea95ddc93" member_type: NON_VOTER last_known_addr { host: "127.27.180.189" port: 45319 } attrs { promote: true } } peers { permanent_uuid: "9149de133657420baaadd406d6f504fd" member_type: NON_VOTER last_known_addr { host: "127.27.180.188" port: 34747 } attrs { promote: true } } }
I20260811 18:55:30.667568 9020 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3781302d19a4a8a9342e98ea95ddc93 [sys.catalog]: This master's current role is: LEARNER
I20260811 18:55:30.667647 9020 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3781302d19a4a8a9342e98ea95ddc93 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "fec531dd027244268997c383dda94895" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "fec531dd027244268997c383dda94895" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 46847 } } peers { permanent_uuid: "e3781302d19a4a8a9342e98ea95ddc93" member_type: NON_VOTER last_known_addr { host: "127.27.180.189" port: 45319 } attrs { promote: true } } peers { permanent_uuid: "9149de133657420baaadd406d6f504fd" member_type: NON_VOTER last_known_addr { host: "127.27.180.188" port: 34747 } attrs { promote: true } } }
I20260811 18:55:30.667706 9020 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3781302d19a4a8a9342e98ea95ddc93 [sys.catalog]: This master's current role is: LEARNER
I20260811 18:55:30.667722 8875 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 9149de133657420baaadd406d6f504fd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:55:30.667824 8875 tablet_replica.cc:330] T 00000000000000000000000000000000 P 9149de133657420baaadd406d6f504fd: stopping tablet replica
I20260811 18:55:30.669059 8859 raft_consensus.cc:1050] T 00000000000000000000000000000000 P fec531dd027244268997c383dda94895: attempting to promote NON_VOTER e3781302d19a4a8a9342e98ea95ddc93 to VOTER
I20260811 18:55:30.669485 8859 consensus_queue.cc:237] T 00000000000000000000000000000000 P fec531dd027244268997c383dda94895 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "fec531dd027244268997c383dda94895" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 46847 } } peers { permanent_uuid: "e3781302d19a4a8a9342e98ea95ddc93" member_type: VOTER last_known_addr { host: "127.27.180.189" port: 45319 } attrs { promote: false } } peers { permanent_uuid: "9149de133657420baaadd406d6f504fd" member_type: NON_VOTER last_known_addr { host: "127.27.180.188" port: 34747 } attrs { promote: true } }
I20260811 18:55:30.670701 8956 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e3781302d19a4a8a9342e98ea95ddc93 [term 1 LEARNER]: Refusing update from remote peer fec531dd027244268997c383dda94895: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260811 18:55:30.671077 8855 consensus_queue.cc:1035] T 00000000000000000000000000000000 P fec531dd027244268997c383dda94895 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3781302d19a4a8a9342e98ea95ddc93" member_type: VOTER last_known_addr { host: "127.27.180.189" port: 45319 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260811 18:55:30.671635 7739 consensus_peers.cc:487] T 00000000000000000000000000000000 P fec531dd027244268997c383dda94895 -> Peer 9149de133657420baaadd406d6f504fd (127.27.180.188:34747): Couldn't send request to peer 9149de133657420baaadd406d6f504fd. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260811 18:55:30.673444 8851 raft_consensus.cc:2932] T 00000000000000000000000000000000 P fec531dd027244268997c383dda94895 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, e3781302d19a4a8a9342e98ea95ddc93 (127.27.180.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "fec531dd027244268997c383dda94895" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 46847 } } peers { permanent_uuid: "e3781302d19a4a8a9342e98ea95ddc93" member_type: VOTER last_known_addr { host: "127.27.180.189" port: 45319 } attrs { promote: false } } peers { permanent_uuid: "9149de133657420baaadd406d6f504fd" member_type: NON_VOTER last_known_addr { host: "127.27.180.188" port: 34747 } attrs { promote: true } } }
I20260811 18:55:30.673964 8956 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e3781302d19a4a8a9342e98ea95ddc93 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, e3781302d19a4a8a9342e98ea95ddc93 (127.27.180.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "fec531dd027244268997c383dda94895" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 46847 } } peers { permanent_uuid: "e3781302d19a4a8a9342e98ea95ddc93" member_type: VOTER last_known_addr { host: "127.27.180.189" port: 45319 } attrs { promote: false } } peers { permanent_uuid: "9149de133657420baaadd406d6f504fd" member_type: NON_VOTER last_known_addr { host: "127.27.180.188" port: 34747 } attrs { promote: true } } }
I20260811 18:55:30.674734 8851 sys_catalog.cc:455] T 00000000000000000000000000000000 P fec531dd027244268997c383dda94895 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "fec531dd027244268997c383dda94895" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "fec531dd027244268997c383dda94895" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 46847 } } peers { permanent_uuid: "e3781302d19a4a8a9342e98ea95ddc93" member_type: VOTER last_known_addr { host: "127.27.180.189" port: 45319 } attrs { promote: false } } peers { permanent_uuid: "9149de133657420baaadd406d6f504fd" member_type: NON_VOTER last_known_addr { host: "127.27.180.188" port: 34747 } attrs { promote: true } } }
I20260811 18:55:30.674835 8851 sys_catalog.cc:458] T 00000000000000000000000000000000 P fec531dd027244268997c383dda94895 [sys.catalog]: This master's current role is: LEADER
I20260811 18:55:30.674938 8851 sys_catalog.cc:455] T 00000000000000000000000000000000 P fec531dd027244268997c383dda94895 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "fec531dd027244268997c383dda94895" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "fec531dd027244268997c383dda94895" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 46847 } } peers { permanent_uuid: "e3781302d19a4a8a9342e98ea95ddc93" member_type: VOTER last_known_addr { host: "127.27.180.189" port: 45319 } attrs { promote: false } } peers { permanent_uuid: "9149de133657420baaadd406d6f504fd" member_type: NON_VOTER last_known_addr { host: "127.27.180.188" port: 34747 } attrs { promote: true } } }
I20260811 18:55:30.674997 8851 sys_catalog.cc:458] T 00000000000000000000000000000000 P fec531dd027244268997c383dda94895 [sys.catalog]: This master's current role is: LEADER
I20260811 18:55:30.675621 9224 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3781302d19a4a8a9342e98ea95ddc93 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "fec531dd027244268997c383dda94895" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "fec531dd027244268997c383dda94895" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 46847 } } peers { permanent_uuid: "e3781302d19a4a8a9342e98ea95ddc93" member_type: VOTER last_known_addr { host: "127.27.180.189" port: 45319 } attrs { promote: false } } peers { permanent_uuid: "9149de133657420baaadd406d6f504fd" member_type: NON_VOTER last_known_addr { host: "127.27.180.188" port: 34747 } attrs { promote: true } } }
I20260811 18:55:30.675807 9224 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3781302d19a4a8a9342e98ea95ddc93 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:55:30.684293 8875 master.cc:552] Master@127.27.180.188:34747 shutdown complete.
I20260811 18:55:30.684342 8875 master_runner.cc:305] Clearing existing system tablet
I20260811 18:55:30.684576 8875 fs_manager.cc:359] Metadata directory not provided
I20260811 18:55:30.684623 8875 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:55:30.686120 8875 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:55:30.687119 9229 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:30.687342 8875 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:55:30.687593 8875 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:55:30.687675 8875 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/master-2/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/master-2/wal
uuid: "9149de133657420baaadd406d6f504fd"
format_stamp: "Formatted at 2026-08-11 18:55:30 on dist-test-slave-2j7r"
server_key: "3675630bcba495c3a5fda1a4cb1bf68a"
server_key_iv: "71b0153d6d6d923206fe7a3509467ec3"
server_key_version: "encryptionkey@0"
I20260811 18:55:30.689865 8875 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 9149de133657420baaadd406d6f504fd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:55:30.692210 8875 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 9149de133657420baaadd406d6f504fd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260811 18:55:30.692359 8875 log.cc:1196] T 00000000000000000000000000000000 P 9149de133657420baaadd406d6f504fd: Deleting WAL directory at /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260811 18:55:30.692656 8875 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 9149de133657420baaadd406d6f504fd: Deleting consensus metadata
I20260811 18:55:30.692855 8875 master_runner.cc:315] Copying system tablet from 127.27.180.190:46847
I20260811 18:55:30.694069 8875 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 9149de133657420baaadd406d6f504fd: tablet copy: Beginning tablet copy session from remote peer at address 127.27.180.190:46847
I20260811 18:55:30.703974 9241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:30.694647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:46847 (local address 127.0.0.1:36148)
0811 18:55:30.694978 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:30.694992 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:30.695120 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:30.695396 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:30.695399 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:30.695611 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:30.695760 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:30.695768 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:30.696814 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:30.696817 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:30.697868 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:30.697880 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:30.698642 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:30.698645 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:30.698747 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:30.699814 (+ 1067us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:30.699829 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:30.700247 (+ 418us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:55:30.702234 (+ 1987us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:30.702238 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:30.702240 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:30.702489 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:30.702845 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:30.702848 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:30.702850 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:30.702944 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:30.703402 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:55:30.703405 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:55:30.703548 (+ 143us) client_negotiation.cc:769] Sending connection context
0811 18:55:30.703652 (+ 104us) client_negotiation.cc:240] Negotiation successful
0811 18:55:30.703802 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":116,"threads_started":1}
I20260811 18:55:30.704035 7828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:30.694742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36148 (local address 127.27.180.190:46847)
0811 18:55:30.694864 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:30.694869 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:30.695072 (+ 203us) server_negotiation.cc:407] Connection header received
0811 18:55:30.695234 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:30.695238 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:30.695289 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:30.695368 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:30.695872 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:30.696691 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:30.698045 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:30.698377 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:30.698463 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:30.700400 (+ 1937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:30.700405 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:30.700408 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:30.700437 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:30.702128 (+ 1691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:30.702630 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:30.702634 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:30.702635 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:30.702735 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:30.703035 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:30.703038 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:30.703040 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:30.703319 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:30.703431 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:30.703709 (+ 278us) server_negotiation.cc:299] Negotiation successful
0811 18:55:30.703828 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 18:55:30.704866 7801 tablet_copy_service.cc:140] P fec531dd027244268997c383dda94895: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 9149de133657420baaadd406d6f504fd ({username='oryx', principal='oryx/127.27.180.188@KRBTEST.COM'} at 127.0.0.1:36148)
I20260811 18:55:30.704968 7801 tablet_copy_service.cc:161] P fec531dd027244268997c383dda94895: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 9149de133657420baaadd406d6f504fd at {username='oryx', principal='oryx/127.27.180.188@KRBTEST.COM'} at 127.0.0.1:36148: session id = 9149de133657420baaadd406d6f504fd-00000000000000000000000000000000
I20260811 18:55:30.706139 7801 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P fec531dd027244268997c383dda94895: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 18:55:30.707455 8875 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260811 18:55:30.709874 8875 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 9149de133657420baaadd406d6f504fd: tablet copy: Starting download of 0 data blocks...
I20260811 18:55:30.710078 8875 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 9149de133657420baaadd406d6f504fd: tablet copy: Starting download of 1 WAL segments...
I20260811 18:55:30.711549 8875 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 9149de133657420baaadd406d6f504fd: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 18:55:30.713235 7801 tablet_copy_service.cc:342] P fec531dd027244268997c383dda94895: Request end of tablet copy session 9149de133657420baaadd406d6f504fd-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.27.180.188@KRBTEST.COM'} at 127.0.0.1:36148
I20260811 18:55:30.713315 7801 tablet_copy_service.cc:433] P fec531dd027244268997c383dda94895: ending tablet copy session 9149de133657420baaadd406d6f504fd-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 9149de133657420baaadd406d6f504fd
I20260811 18:55:30.720518 8875 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:55:30.722224 9242 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:30.725982 9243 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:55:30.726117 8875 server_base.cc:1034] running on GCE node
W20260811 18:55:30.726291 9246 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:55:30.726517 8875 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:55:30.726832 8875 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:55:30.727979 8875 hybrid_clock.cc:648] HybridClock initialized: now 1786474530727965 us; error 28 us; skew 500 ppm
Aug 11 18:55:30 dist-test-slave-2j7r krb5kdc[7666](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474530, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.180.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:55:30.729696 8875 init.cc:377] Logged in from keytab as oryx/127.27.180.188@KRBTEST.COM (short username oryx)
I20260811 18:55:30.730162 8875 webserver.cc:460] Webserver started at http://127.27.180.188:34467/ using document root <none> and password file <none>
I20260811 18:55:30.730342 8875 fs_manager.cc:359] Metadata directory not provided
I20260811 18:55:30.730387 8875 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:55:30.731655 8875 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:55:30.732710 9259 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:30.733011 8875 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 18:55:30.733108 8875 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/master-2/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/master-2/wal
uuid: "9149de133657420baaadd406d6f504fd"
format_stamp: "Formatted at 2026-08-11 18:55:30 on dist-test-slave-2j7r"
server_key: "3675630bcba495c3a5fda1a4cb1bf68a"
server_key_iv: "71b0153d6d6d923206fe7a3509467ec3"
server_key_version: "encryptionkey@0"
I20260811 18:55:30.733194 8875 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:55:30.741355 8875 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:55:30.743561 8875 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:55:30.748729 8875 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.188:34747
I20260811 18:55:30.748765 9359 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.188:34747 every 8 connection(s)
I20260811 18:55:30.749370 8875 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/master-2/data/info.pb
I20260811 18:55:30.751171 9362 sys_catalog.cc:263] Verifying existing consensus state
I20260811 18:55:30.752296 9362 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9149de133657420baaadd406d6f504fd: Bootstrap starting.
I20260811 18:55:30.753796 8374 heartbeater.cc:503] Master 127.27.180.190:46847 was elected leader, sending a full tablet report...
I20260811 18:55:30.763504 9362 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9149de133657420baaadd406d6f504fd: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 18:55:30.763896 9362 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9149de133657420baaadd406d6f504fd: Bootstrap complete.
I20260811 18:55:30.764575 9362 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9149de133657420baaadd406d6f504fd [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "fec531dd027244268997c383dda94895" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 46847 } } peers { permanent_uuid: "e3781302d19a4a8a9342e98ea95ddc93" member_type: VOTER last_known_addr { host: "127.27.180.189" port: 45319 } attrs { promote: false } } peers { permanent_uuid: "9149de133657420baaadd406d6f504fd" member_type: NON_VOTER last_known_addr { host: "127.27.180.188" port: 34747 } attrs { promote: true } }
I20260811 18:55:30.764797 9362 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9149de133657420baaadd406d6f504fd [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 9149de133657420baaadd406d6f504fd, State: Initialized, Role: LEARNER
I20260811 18:55:30.764955 9362 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9149de133657420baaadd406d6f504fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "fec531dd027244268997c383dda94895" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 46847 } } peers { permanent_uuid: "e3781302d19a4a8a9342e98ea95ddc93" member_type: VOTER last_known_addr { host: "127.27.180.189" port: 45319 } attrs { promote: false } } peers { permanent_uuid: "9149de133657420baaadd406d6f504fd" member_type: NON_VOTER last_known_addr { host: "127.27.180.188" port: 34747 } attrs { promote: true } }
I20260811 18:55:30.765399 9370 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9149de133657420baaadd406d6f504fd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "fec531dd027244268997c383dda94895" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 46847 } } peers { permanent_uuid: "e3781302d19a4a8a9342e98ea95ddc93" member_type: VOTER last_known_addr { host: "127.27.180.189" port: 45319 } attrs { promote: false } } peers { permanent_uuid: "9149de133657420baaadd406d6f504fd" member_type: NON_VOTER last_known_addr { host: "127.27.180.188" port: 34747 } attrs { promote: true } } }
I20260811 18:55:30.765522 9370 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9149de133657420baaadd406d6f504fd [sys.catalog]: This master's current role is: LEARNER
I20260811 18:55:30.765646 9362 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9149de133657420baaadd406d6f504fd [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:55:30.769531 9391 catalog_manager.cc:1259] Loaded cluster ID: c733e261438841f4aa5d657b05032fa9
I20260811 18:55:30.769583 9391 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9149de133657420baaadd406d6f504fd: loading cluster ID for follower catalog manager: success
I20260811 18:55:30.771698 9391 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9149de133657420baaadd406d6f504fd: acquiring CA information for follower catalog manager: success
I20260811 18:55:30.773061 9391 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9149de133657420baaadd406d6f504fd: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260811 18:55:30.784904 9215 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:55:30.785145 9215 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:55:30.785195 9215 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:55:30.789149 9215 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:55:30.789194 9215 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:55:30.789247 9215 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.180.129
I20260811 18:55:30.794158 9215 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:33071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/krb5kdc/oryx_127.27.180.129.keytab
--principal=oryx/127.27.180.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.180.129:32985
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.180.129
--webserver_port=35595
--tserver_master_addrs=127.27.180.190:46847,127.27.180.189:45319,127.27.180.188:34747
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9215
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.180.129
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:55:30.795033 9215 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:55:30.795917 9215 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:55:30.805285 9397 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:30.805281 9405 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:30.805277 9400 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:55:30.805401 9215 server_base.cc:1034] running on GCE node
I20260811 18:55:30.805821 9215 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:55:30.806414 9215 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:55:30.807567 9215 hybrid_clock.cc:648] HybridClock initialized: now 1786474530807548 us; error 30 us; skew 500 ppm
Aug 11 18:55:30 dist-test-slave-2j7r krb5kdc[7666](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474530, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.180.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:55:30.810214 9215 init.cc:377] Logged in from keytab as oryx/127.27.180.129@KRBTEST.COM (short username oryx)
I20260811 18:55:30.810992 9215 webserver.cc:460] Webserver started at http://127.27.180.129:35595/ using document root <none> and password file <none>
I20260811 18:55:30.811493 9215 fs_manager.cc:359] Metadata directory not provided
I20260811 18:55:30.811574 9215 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:55:30.815240 9215 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260811 18:55:30.817205 9420 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:30.818308 9215 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 18:55:30.818429 9215 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/ts-0/wal
uuid: "b1221343a5f8419ab3f58e8e328cc30c"
format_stamp: "Formatted at 2026-08-11 18:55:29 on dist-test-slave-2j7r"
server_key: "55dcc668bcb1e2524a3ff8107dd43d9e"
server_key_iv: "f1b2aba774940062536ef7ba941ebd88"
server_key_version: "encryptionkey@0"
I20260811 18:55:30.818899 9215 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:55:30.830354 9215 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:55:30.833205 9215 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:55:30.833365 9215 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:55:30.833896 9215 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:55:30.834726 9215 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:55:30.834772 9215 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:30.834822 9215 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:55:30.834851 9215 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:30.844223 9215 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.129:32985
I20260811 18:55:30.844257 9632 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.129:32985 every 8 connection(s)
I20260811 18:55:30.845160 9215 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/ts-0/data/info.pb
I20260811 18:55:30.850804 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 9215
I20260811 18:55:30.850991 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 8116
Aug 11 18:55:30 dist-test-slave-2j7r krb5kdc[7666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474530, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.180.129@KRBTEST.COM for oryx/127.27.180.189@KRBTEST.COM
Aug 11 18:55:30 dist-test-slave-2j7r krb5kdc[7666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474530, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.180.129@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM
Aug 11 18:55:30 dist-test-slave-2j7r krb5kdc[7666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474530, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.180.129@KRBTEST.COM for oryx/127.27.180.188@KRBTEST.COM
I20260811 18:55:30.899544 9179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:30.846830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.129:37933 (local address 127.27.180.189:45319)
0811 18:55:30.846954 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:30.846958 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:30.848619 (+ 1661us) server_negotiation.cc:407] Connection header received
0811 18:55:30.849175 (+ 556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:30.849178 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:30.849243 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:30.849320 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:30.850681 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:30.851443 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:30.882149 (+ 30706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:30.882448 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:30.882518 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:30.895383 (+ 12865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:30.895388 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:30.895391 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:30.895425 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:30.896976 (+ 1551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:30.897658 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:30.897662 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:30.897664 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:30.897745 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:30.898186 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:30.898190 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:30.898192 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:30.898424 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:30.898512 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:30.899240 (+ 728us) server_negotiation.cc:299] Negotiation successful
0811 18:55:30.899369 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 18:55:30.899858 7828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:30.847136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.129:47967 (local address 127.27.180.190:46847)
0811 18:55:30.847277 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:30.847282 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:30.848406 (+ 1124us) server_negotiation.cc:407] Connection header received
0811 18:55:30.849208 (+ 802us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:30.849211 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:30.849280 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:30.849333 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:30.850667 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:30.851386 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:30.892570 (+ 41184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:30.892822 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:30.892939 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:30.895951 (+ 3012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:30.895956 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:30.895959 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:30.895993 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:30.897580 (+ 1587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:30.898270 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:30.898274 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:30.898275 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:30.898363 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:30.898885 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:30.898889 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:30.898890 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:30.899119 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:30.899210 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:30.899576 (+ 366us) server_negotiation.cc:299] Negotiation successful
0811 18:55:30.899712 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 18:55:30.900607 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.180.130:45897
--local_ip_for_outbound_sockets=127.27.180.130
--tserver_master_addrs=127.27.180.190:46847
--webserver_port=33069
--webserver_interface=127.27.180.130
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/krb5kdc/oryx_127.27.180.130.keytab
--principal=oryx/127.27.180.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:33071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.27.180.190:46847,127.27.180.189:45319
--tserver_master_addrs=127.27.180.190:46847,127.27.180.189:45319,127.27.180.188:34747 with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 18:55:30.901263 9641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:30.847258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.188:34747 (local address 127.27.180.129:33989)
0811 18:55:30.847940 (+ 682us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:30.847989 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:30.848955 (+ 966us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:30.849507 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:30.849513 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:30.849966 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:30.850502 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:30.850517 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:30.893955 (+ 43438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:30.893959 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:30.894796 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:30.894809 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:30.895541 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:30.895545 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:30.895641 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:30.896083 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:30.896098 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:30.897468 (+ 1370us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:55:30.899399 (+ 1931us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:30.899401 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:30.899403 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:30.899875 (+ 472us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:30.900172 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:30.900175 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:30.900177 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:30.900280 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:30.900716 (+ 436us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:55:30.900719 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:55:30.900802 (+ 83us) client_negotiation.cc:769] Sending connection context
0811 18:55:30.900896 (+ 94us) client_negotiation.cc:240] Negotiation successful
0811 18:55:30.901013 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":493,"mutex_wait_us":57,"spinlock_wait_cycles":12672,"thread_start_us":656,"threads_started":1}
I20260811 18:55:30.901517 9647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:30.847258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.189:45319 (local address 127.27.180.129:37933)
0811 18:55:30.848534 (+ 1276us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:30.848545 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:30.848957 (+ 412us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:30.849505 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:30.849512 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:30.849965 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:30.850500 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:30.850512 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:30.880673 (+ 30161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:30.880677 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:30.881973 (+ 1296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:30.881990 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:30.882751 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:30.882754 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:30.882972 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:30.892711 (+ 9739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:30.892739 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:30.895140 (+ 2401us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:55:30.897140 (+ 2000us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:30.897148 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:30.897168 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:30.897553 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:30.897874 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:30.897877 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:30.897879 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:30.898076 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:30.898578 (+ 502us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:55:30.898583 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:55:30.898946 (+ 363us) client_negotiation.cc:769] Sending connection context
0811 18:55:30.899723 (+ 777us) client_negotiation.cc:240] Negotiation successful
0811 18:55:30.900097 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1200,"mutex_wait_us":201,"spinlock_wait_cycles":24960,"thread_start_us":115,"threads_started":1}
I20260811 18:55:30.901266 9646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:30.847258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:46847 (local address 127.27.180.129:47967)
0811 18:55:30.848300 (+ 1042us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:30.848312 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:30.848955 (+ 643us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:30.849507 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:30.849513 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:30.849969 (+ 456us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:30.850503 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:30.850517 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:30.891444 (+ 40927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:30.891447 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:30.892380 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:30.892397 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:30.892979 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:30.892982 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:30.893110 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:30.894324 (+ 1214us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:30.894340 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:30.895741 (+ 1401us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:55:30.897704 (+ 1963us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:30.897707 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:30.897709 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:30.898160 (+ 451us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:30.898475 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:30.898478 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:30.898480 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:30.898777 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:30.899274 (+ 497us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:55:30.899277 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:55:30.899370 (+ 93us) client_negotiation.cc:769] Sending connection context
0811 18:55:30.900045 (+ 675us) client_negotiation.cc:240] Negotiation successful
0811 18:55:30.900191 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":949,"mutex_wait_us":82,"spinlock_wait_cycles":35200,"thread_start_us":168,"threads_started":1}
I20260811 18:55:30.902565 9639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:30.847448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.129:33989 (local address 127.27.180.188:34747)
0811 18:55:30.847692 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:30.847696 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:30.848663 (+ 967us) server_negotiation.cc:407] Connection header received
0811 18:55:30.849268 (+ 605us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:30.849271 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:30.849323 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:30.849395 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:30.850667 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:30.893802 (+ 43135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:30.894953 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:30.895294 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:30.895391 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:30.897630 (+ 2239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:30.897634 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:30.897637 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:30.897663 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:30.899286 (+ 1623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:30.899980 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:30.899984 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:30.899985 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:30.900075 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:30.900379 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:30.900382 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:30.900383 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:30.900590 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:30.900674 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:30.901762 (+ 1088us) server_negotiation.cc:299] Negotiation successful
0811 18:55:30.902109 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":78,"threads_started":1}
I20260811 18:55:30.902772 9635 heartbeater.cc:347] Connected to a master server at 127.27.180.189:45319
I20260811 18:55:30.902962 9634 heartbeater.cc:347] Connected to a master server at 127.27.180.190:46847
I20260811 18:55:30.903083 9634 heartbeater.cc:464] Registering TS with master...
I20260811 18:55:30.903086 9635 heartbeater.cc:464] Registering TS with master...
I20260811 18:55:30.903366 9633 heartbeater.cc:347] Connected to a master server at 127.27.180.188:34747
I20260811 18:55:30.903451 9633 heartbeater.cc:464] Registering TS with master...
I20260811 18:55:30.903704 9634 heartbeater.cc:511] Master 127.27.180.190:46847 requested a full tablet report, sending...
I20260811 18:55:30.903729 9635 heartbeater.cc:511] Master 127.27.180.189:45319 requested a full tablet report, sending...
I20260811 18:55:30.903704 9633 heartbeater.cc:511] Master 127.27.180.188:34747 requested a full tablet report, sending...
I20260811 18:55:30.904975 7768 ts_manager.cc:194] Re-registered known tserver with Master: b1221343a5f8419ab3f58e8e328cc30c (127.27.180.129:32985)
I20260811 18:55:30.905251 9278 ts_manager.cc:194] Registered new tserver with Master: b1221343a5f8419ab3f58e8e328cc30c (127.27.180.129:32985)
I20260811 18:55:30.905287 8924 ts_manager.cc:194] Registered new tserver with Master: b1221343a5f8419ab3f58e8e328cc30c (127.27.180.129:32985)
I20260811 18:55:30.905726 7768 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.27.180.129@KRBTEST.COM'} at 127.27.180.129:47967
I20260811 18:55:30.965242 8647 heartbeater.cc:503] Master 127.27.180.190:46847 was elected leader, sending a full tablet report...
W20260811 18:55:31.011564 9648 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:55:31.011802 9648 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:55:31.011868 9648 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:55:31.015800 9648 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:55:31.015870 9648 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:55:31.015944 9648 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.180.130
I20260811 18:55:31.020993 9648 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:33071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/krb5kdc/oryx_127.27.180.130.keytab
--principal=oryx/127.27.180.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.180.130:45897
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.180.130
--webserver_port=33069
--tserver_master_addrs=127.27.180.190:46847,127.27.180.189:45319,127.27.180.188:34747
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9648
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.180.130
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:55:31.021973 9648 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:55:31.022948 9648 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:55:31.032176 9659 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:31.032189 9655 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:55:31.032292 9648 server_base.cc:1034] running on GCE node
W20260811 18:55:31.032189 9662 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:55:31.032855 9648 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:55:31.033509 9648 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:55:31.034683 9648 hybrid_clock.cc:648] HybridClock initialized: now 1786474531034659 us; error 36 us; skew 500 ppm
Aug 11 18:55:31 dist-test-slave-2j7r krb5kdc[7666](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474531, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.180.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:55:31.037343 9648 init.cc:377] Logged in from keytab as oryx/127.27.180.130@KRBTEST.COM (short username oryx)
I20260811 18:55:31.038177 9648 webserver.cc:460] Webserver started at http://127.27.180.130:33069/ using document root <none> and password file <none>
I20260811 18:55:31.038661 9648 fs_manager.cc:359] Metadata directory not provided
I20260811 18:55:31.038726 9648 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:55:31.042366 9648 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260811 18:55:31.044242 9678 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:31.045468 9648 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 18:55:31.045572 9648 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/ts-1/wal
uuid: "485e9ca42b2a45aa9aeeb564daf0edda"
format_stamp: "Formatted at 2026-08-11 18:55:29 on dist-test-slave-2j7r"
server_key: "e1227394f9b3fa0d1993f35aaa5a82df"
server_key_iv: "38383799bfbbf3f8a0fe6bcabfa42ea1"
server_key_version: "encryptionkey@0"
I20260811 18:55:31.046090 9648 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:55:31.069944 9648 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:55:31.072981 9648 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:55:31.073175 9648 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:55:31.073748 9648 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:55:31.074613 9648 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:55:31.074671 9648 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:31.074721 9648 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:55:31.074759 9648 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:31.083102 9648 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.130:45897
I20260811 18:55:31.083159 9865 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.130:45897 every 8 connection(s)
I20260811 18:55:31.084020 9648 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/ts-1/data/info.pb
I20260811 18:55:31.088439 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 9648
I20260811 18:55:31.088615 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 8383
Aug 11 18:55:31 dist-test-slave-2j7r krb5kdc[7666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474531, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.180.130@KRBTEST.COM for oryx/127.27.180.189@KRBTEST.COM
I20260811 18:55:31.097738 9179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:31.085694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.130:50041 (local address 127.27.180.189:45319)
0811 18:55:31.085820 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:31.085839 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:31.088039 (+ 2200us) server_negotiation.cc:407] Connection header received
0811 18:55:31.088295 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:31.088299 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:31.088355 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:31.088410 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:31.089884 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:31.090393 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:31.091625 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:31.091903 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:31.091962 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:31.094424 (+ 2462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:31.094428 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:31.094430 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:31.094454 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:31.095802 (+ 1348us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:31.096296 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:31.096298 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:31.096299 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:31.096372 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:31.096649 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:31.096651 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:31.096652 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:31.096864 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:31.096929 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:31.097514 (+ 585us) server_negotiation.cc:299] Negotiation successful
0811 18:55:31.097616 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 18:55:31.098989 9873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:31.087619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.189:45319 (local address 127.27.180.130:50041)
0811 18:55:31.087951 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:31.087964 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:31.088079 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:31.088802 (+ 723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:31.088809 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:31.089232 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:31.089744 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:31.089756 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:31.090517 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:31.090519 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:31.091500 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:31.091512 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:31.092125 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:31.092128 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:31.092282 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:31.092725 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:31.092741 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:31.094231 (+ 1490us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:55:31.095927 (+ 1696us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:31.095933 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:31.095946 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:31.096201 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:31.096453 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:31.096455 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:31.096456 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:31.096573 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:31.097002 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:55:31.097008 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:55:31.097287 (+ 279us) client_negotiation.cc:769] Sending connection context
0811 18:55:31.097881 (+ 594us) client_negotiation.cc:240] Negotiation successful
0811 18:55:31.098132 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":98,"threads_started":1}
Aug 11 18:55:31 dist-test-slave-2j7r krb5kdc[7666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474531, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.180.130@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM
I20260811 18:55:31.122078 9869 heartbeater.cc:347] Connected to a master server at 127.27.180.189:45319
I20260811 18:55:31.122375 9869 heartbeater.cc:464] Registering TS with master...
I20260811 18:55:31.123044 9869 heartbeater.cc:511] Master 127.27.180.189:45319 requested a full tablet report, sending...
I20260811 18:55:31.124135 7828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:31.087521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.130:49789 (local address 127.27.180.190:46847)
0811 18:55:31.087632 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:31.087636 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:31.087648 (+ 12us) server_negotiation.cc:407] Connection header received
0811 18:55:31.088314 (+ 666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:31.088317 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:31.088363 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:31.088410 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:31.114606 (+ 26196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:31.115458 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:31.116785 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:31.117265 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:31.117399 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:31.120591 (+ 3192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:31.120594 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:31.120597 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:31.120622 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:31.122242 (+ 1620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:31.122799 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:31.122803 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:31.122804 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:31.122900 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:31.123210 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:31.123214 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:31.123215 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:31.123428 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:31.123499 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:31.123859 (+ 360us) server_negotiation.cc:299] Negotiation successful
0811 18:55:31.123982 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 18:55:31.124645 9872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:31.086174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:46847 (local address 127.27.180.130:49789)
0811 18:55:31.086633 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:31.086684 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:31.088080 (+ 1396us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:31.114055 (+ 25975us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:31.114060 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:31.114303 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:31.114427 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:31.114434 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:31.115687 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:31.115705 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:31.116622 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:31.116633 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:31.117571 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:31.117577 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:31.117694 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:31.118805 (+ 1111us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:31.118823 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:31.120445 (+ 1622us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:55:31.122370 (+ 1925us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:31.122372 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:31.122374 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:31.122594 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:31.123012 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:31.123014 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:31.123016 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:31.123104 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:31.123568 (+ 464us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:55:31.123570 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:55:31.123660 (+ 90us) client_negotiation.cc:769] Sending connection context
0811 18:55:31.124334 (+ 674us) client_negotiation.cc:240] Negotiation successful
0811 18:55:31.124474 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":114,"threads_started":1}
I20260811 18:55:31.124688 8924 ts_manager.cc:194] Registered new tserver with Master: 485e9ca42b2a45aa9aeeb564daf0edda (127.27.180.130:45897)
I20260811 18:55:31.125237 9867 heartbeater.cc:347] Connected to a master server at 127.27.180.190:46847
I20260811 18:55:31.125334 9867 heartbeater.cc:464] Registering TS with master...
I20260811 18:55:31.125519 9867 heartbeater.cc:511] Master 127.27.180.190:46847 requested a full tablet report, sending...
I20260811 18:55:31.126153 7768 ts_manager.cc:194] Re-registered known tserver with Master: 485e9ca42b2a45aa9aeeb564daf0edda (127.27.180.130:45897)
Aug 11 18:55:31 dist-test-slave-2j7r krb5kdc[7666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474531, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.180.130@KRBTEST.COM for oryx/127.27.180.188@KRBTEST.COM
I20260811 18:55:31.126669 7768 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.27.180.130@KRBTEST.COM'} at 127.27.180.130:49789
I20260811 18:55:31.128671 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.180.131:39783
--local_ip_for_outbound_sockets=127.27.180.131
--tserver_master_addrs=127.27.180.190:46847
--webserver_port=37389
--webserver_interface=127.27.180.131
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/krb5kdc/oryx_127.27.180.131.keytab
--principal=oryx/127.27.180.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:33071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.27.180.190:46847,127.27.180.189:45319
--tserver_master_addrs=127.27.180.190:46847,127.27.180.189:45319,127.27.180.188:34747 with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 18:55:31.130095 9639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:31.087282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.130:51527 (local address 127.27.180.188:34747)
0811 18:55:31.087399 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:31.087403 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:31.087926 (+ 523us) server_negotiation.cc:407] Connection header received
0811 18:55:31.088315 (+ 389us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:31.088319 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:31.088381 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:31.088437 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:31.121875 (+ 33438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:31.122652 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:31.123982 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:31.124326 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:31.124398 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:31.127026 (+ 2628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:31.127030 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:31.127033 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:31.127063 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:31.128505 (+ 1442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:31.128912 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:31.128915 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:31.128917 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:31.129005 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:31.129310 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:31.129313 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:31.129315 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:31.129512 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:31.129608 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:31.129822 (+ 214us) server_negotiation.cc:299] Negotiation successful
0811 18:55:31.129951 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 18:55:31.130168 9875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:31.087199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.188:34747 (local address 127.27.180.130:51527)
0811 18:55:31.087839 (+ 640us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:31.087849 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:31.088084 (+ 235us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:31.121399 (+ 33315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:31.121403 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:31.121578 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:31.121701 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:31.121709 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:31.122879 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:31.122881 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:31.123838 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:31.123849 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:31.124946 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:31.124950 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:31.125034 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:31.125477 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:31.125490 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:31.126893 (+ 1403us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:55:31.128603 (+ 1710us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:31.128605 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:31.128607 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:31.128781 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:31.129109 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:31.129113 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:31.129115 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:31.129190 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:31.129594 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:55:31.129596 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:55:31.129666 (+ 70us) client_negotiation.cc:769] Sending connection context
0811 18:55:31.129737 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 18:55:31.129845 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":587,"thread_start_us":106,"threads_started":1}
I20260811 18:55:31.130815 9866 heartbeater.cc:347] Connected to a master server at 127.27.180.188:34747
I20260811 18:55:31.130903 9866 heartbeater.cc:464] Registering TS with master...
I20260811 18:55:31.131073 9866 heartbeater.cc:511] Master 127.27.180.188:34747 requested a full tablet report, sending...
I20260811 18:55:31.131731 9278 ts_manager.cc:194] Registered new tserver with Master: 485e9ca42b2a45aa9aeeb564daf0edda (127.27.180.130:45897)
I20260811 18:55:31.200106 9880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:31.196016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.188:34747 (local address 127.0.0.1:54696)
0811 18:55:31.196334 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:31.196349 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:31.196461 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:31.196789 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:31.196792 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:31.196814 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:55:31.196934 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:31.196940 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:31.198000 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:31.198003 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:31.198772 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:31.198781 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:31.199722 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:31.199725 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:31.199812 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:31.199836 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 18:55:31.199887 (+ 51us) client_negotiation.cc:240] Negotiation successful
0811 18:55:31.199965 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":85,"threads_started":1}
I20260811 18:55:31.200131 9639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:31.196118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54696 (local address 127.27.180.188:34747)
0811 18:55:31.196273 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:31.196279 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:31.196422 (+ 143us) server_negotiation.cc:407] Connection header received
0811 18:55:31.196557 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:31.196562 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:31.196636 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:31.196746 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:55:31.197036 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:31.197891 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:31.198928 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:31.199597 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:31.199725 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:31.199873 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:31.199920 (+ 47us) server_negotiation.cc:299] Negotiation successful
0811 18:55:31.199995 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 18:55:31.200762 9316 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 9149de133657420baaadd406d6f504fd [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260811 18:55:31.201192 9370 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9149de133657420baaadd406d6f504fd [sys.catalog]: SysCatalogTable state changed. Reason: New leader fec531dd027244268997c383dda94895. Latest consensus state: current_term: 1 leader_uuid: "fec531dd027244268997c383dda94895" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "fec531dd027244268997c383dda94895" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 46847 } } peers { permanent_uuid: "e3781302d19a4a8a9342e98ea95ddc93" member_type: VOTER last_known_addr { host: "127.27.180.189" port: 45319 } attrs { promote: false } } peers { permanent_uuid: "9149de133657420baaadd406d6f504fd" member_type: NON_VOTER last_known_addr { host: "127.27.180.188" port: 34747 } attrs { promote: true } } }
I20260811 18:55:31.201349 9370 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9149de133657420baaadd406d6f504fd [sys.catalog]: This master's current role is: LEARNER
W20260811 18:55:31.241683 9877 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:55:31.241946 9877 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:55:31.242010 9877 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:55:31.246268 9877 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:55:31.246353 9877 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:55:31.246438 9877 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.180.131
I20260811 18:55:31.251967 9877 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:33071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/krb5kdc/oryx_127.27.180.131.keytab
--principal=oryx/127.27.180.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.180.131:39783
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.180.131
--webserver_port=37389
--tserver_master_addrs=127.27.180.190:46847,127.27.180.189:45319,127.27.180.188:34747
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9877
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.180.131
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:55:31.253050 9877 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:55:31.254170 9877 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:55:31.264492 9888 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:31.264545 9896 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:31.264501 9890 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:55:31.264797 9877 server_base.cc:1034] running on GCE node
I20260811 18:55:31.265334 9877 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:55:31.265967 9877 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:55:31.267122 9877 hybrid_clock.cc:648] HybridClock initialized: now 1786474531267084 us; error 52 us; skew 500 ppm
Aug 11 18:55:31 dist-test-slave-2j7r krb5kdc[7666](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474531, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.180.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:55:31.270136 9877 init.cc:377] Logged in from keytab as oryx/127.27.180.131@KRBTEST.COM (short username oryx)
I20260811 18:55:31.270994 9877 webserver.cc:460] Webserver started at http://127.27.180.131:37389/ using document root <none> and password file <none>
I20260811 18:55:31.271592 9877 fs_manager.cc:359] Metadata directory not provided
I20260811 18:55:31.271677 9877 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:55:31.275594 9877 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260811 18:55:31.277595 9907 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:31.278795 9877 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 18:55:31.278913 9877 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/ts-2/wal
uuid: "93dbcfa6e0784096bed84ddb3f611598"
format_stamp: "Formatted at 2026-08-11 18:55:29 on dist-test-slave-2j7r"
server_key: "15df247d31229e753373e60c424bf69f"
server_key_iv: "21bca37456fbe576c9dc8680a4a72ff5"
server_key_version: "encryptionkey@0"
I20260811 18:55:31.279422 9877 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:55:31.291693 9877 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:55:31.294390 9877 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:55:31.294557 9877 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:55:31.295130 9877 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:55:31.295939 9877 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:55:31.295985 9877 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:31.296037 9877 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:55:31.296067 9877 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:31.304941 9877 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.131:39783
I20260811 18:55:31.304991 10129 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.131:39783 every 8 connection(s)
I20260811 18:55:31.305960 9877 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/ts-2/data/info.pb
I20260811 18:55:31.315508 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 9877
Aug 11 18:55:31 dist-test-slave-2j7r krb5kdc[7666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474531, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.180.131@KRBTEST.COM for oryx/127.27.180.189@KRBTEST.COM
Aug 11 18:55:31 dist-test-slave-2j7r krb5kdc[7666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474531, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.180.131@KRBTEST.COM for oryx/127.27.180.188@KRBTEST.COM
Aug 11 18:55:31 dist-test-slave-2j7r krb5kdc[7666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474531, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.180.131@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM
I20260811 18:55:31.322407 9639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:31.309791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.131:33117 (local address 127.27.180.188:34747)
0811 18:55:31.309925 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:31.309929 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:31.309944 (+ 15us) server_negotiation.cc:407] Connection header received
0811 18:55:31.310357 (+ 413us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:31.310361 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:31.310434 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:31.310510 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:31.312460 (+ 1950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:31.313265 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:31.314114 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:31.314435 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:31.314522 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:31.317748 (+ 3226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:31.317752 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:31.317755 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:31.317783 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:31.319754 (+ 1971us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:31.320362 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:31.320366 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:31.320368 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:31.320458 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:31.320894 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:31.320897 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:31.320899 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:31.321137 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:31.321227 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:31.322132 (+ 905us) server_negotiation.cc:299] Negotiation successful
0811 18:55:31.322250 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 18:55:31.323313 10138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:31.309350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.188:34747 (local address 127.27.180.131:33117)
0811 18:55:31.309629 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:31.309642 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:31.310152 (+ 510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:31.310690 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:31.310697 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:31.311107 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:31.311637 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:31.311650 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:31.313397 (+ 1747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:31.313401 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:31.313997 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:31.314006 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:31.314638 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:31.314645 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:31.314746 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:31.315085 (+ 339us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:31.315097 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:31.317558 (+ 2461us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:55:31.319856 (+ 2298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:31.319859 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:31.319861 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:31.320250 (+ 389us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:31.320560 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:31.320563 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:31.320565 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:31.320782 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:31.321287 (+ 505us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:55:31.321294 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:55:31.321676 (+ 382us) client_negotiation.cc:769] Sending connection context
0811 18:55:31.322003 (+ 327us) client_negotiation.cc:240] Negotiation successful
0811 18:55:31.322968 (+ 965us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"mutex_wait_us":840,"spinlock_wait_cycles":58880,"thread_start_us":104,"threads_started":1}
I20260811 18:55:31.323729 7828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:31.307742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.131:57505 (local address 127.27.180.190:46847)
0811 18:55:31.307885 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:31.307890 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:31.309234 (+ 1344us) server_negotiation.cc:407] Connection header received
0811 18:55:31.310382 (+ 1148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:31.310385 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:31.310440 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:31.310491 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:31.311781 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:31.312309 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:31.313518 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:31.313841 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:31.313938 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:31.317750 (+ 3812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:31.317754 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:31.317756 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:31.317786 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:31.320094 (+ 2308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:31.320576 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:31.320580 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:31.320581 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:31.320667 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:31.320965 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:31.320968 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:31.320970 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:31.321216 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:31.321315 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:31.323466 (+ 2151us) server_negotiation.cc:299] Negotiation successful
0811 18:55:31.323583 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 18:55:31.323740 9179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:31.309941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.131:40683 (local address 127.27.180.189:45319)
0811 18:55:31.310047 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:31.310050 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:31.310062 (+ 12us) server_negotiation.cc:407] Connection header received
0811 18:55:31.310382 (+ 320us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:31.310385 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:31.310440 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:31.310491 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:31.311801 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:31.312319 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:31.313619 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:31.313946 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:31.314012 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:31.317748 (+ 3736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:31.317752 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:31.317755 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:31.317783 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:31.319657 (+ 1874us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:31.320362 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:31.320366 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:31.320368 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:31.320458 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:31.320894 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:31.320897 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:31.320899 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:31.321137 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:31.321227 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:31.323479 (+ 2252us) server_negotiation.cc:299] Negotiation successful
0811 18:55:31.323600 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260811 18:55:31.323315 10137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:31.308174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:46847 (local address 127.27.180.131:57505)
0811 18:55:31.309057 (+ 883us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:31.309129 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:31.310151 (+ 1022us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:31.310692 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:31.310698 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:31.311136 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:31.311639 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:31.311653 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:31.312448 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:31.312451 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:31.313405 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:31.313414 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:31.313986 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:31.313990 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:31.314160 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:31.314720 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:31.314741 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:31.317559 (+ 2818us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:55:31.320204 (+ 2645us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:31.320207 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:31.320209 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:31.320460 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:31.320772 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:31.320775 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:31.320777 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:31.320866 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:31.321445 (+ 579us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:55:31.321448 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:55:31.321709 (+ 261us) client_negotiation.cc:769] Sending connection context
0811 18:55:31.322587 (+ 878us) client_negotiation.cc:240] Negotiation successful
0811 18:55:31.322718 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":687,"mutex_wait_us":196,"spinlock_wait_cycles":18048,"thread_start_us":534,"threads_started":1}
I20260811 18:55:31.323313 10140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:31.309595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.189:45319 (local address 127.27.180.131:40683)
0811 18:55:31.309928 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:31.309941 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:31.310159 (+ 218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:31.310692 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:31.310699 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:31.311138 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:31.311639 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:31.311653 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:31.312459 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:31.312462 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:31.313490 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:31.313502 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:31.314162 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:31.314165 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:31.314257 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:31.314862 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:31.314875 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:31.317558 (+ 2683us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:55:31.319793 (+ 2235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:31.319800 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:31.319816 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:31.320249 (+ 433us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:31.320560 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:31.320563 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:31.320565 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:31.320782 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:31.321287 (+ 505us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:55:31.321294 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:55:31.321676 (+ 382us) client_negotiation.cc:769] Sending connection context
0811 18:55:31.321895 (+ 219us) client_negotiation.cc:240] Negotiation successful
0811 18:55:31.322260 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"mutex_wait_us":144,"spinlock_wait_cycles":24192,"thread_start_us":107,"threads_started":1}
I20260811 18:55:31.324751 10131 heartbeater.cc:347] Connected to a master server at 127.27.180.188:34747
I20260811 18:55:31.325137 10131 heartbeater.cc:464] Registering TS with master...
I20260811 18:55:31.325454 10135 heartbeater.cc:347] Connected to a master server at 127.27.180.189:45319
I20260811 18:55:31.325542 10135 heartbeater.cc:464] Registering TS with master...
I20260811 18:55:31.325711 10131 heartbeater.cc:511] Master 127.27.180.188:34747 requested a full tablet report, sending...
I20260811 18:55:31.325714 10135 heartbeater.cc:511] Master 127.27.180.189:45319 requested a full tablet report, sending...
I20260811 18:55:31.324877 10132 heartbeater.cc:347] Connected to a master server at 127.27.180.190:46847
I20260811 18:55:31.326256 10132 heartbeater.cc:464] Registering TS with master...
I20260811 18:55:31.326419 10132 heartbeater.cc:511] Master 127.27.180.190:46847 requested a full tablet report, sending...
I20260811 18:55:31.326838 9278 ts_manager.cc:194] Registered new tserver with Master: 93dbcfa6e0784096bed84ddb3f611598 (127.27.180.131:39783)
I20260811 18:55:31.326848 8924 ts_manager.cc:194] Registered new tserver with Master: 93dbcfa6e0784096bed84ddb3f611598 (127.27.180.131:39783)
I20260811 18:55:31.327436 7768 ts_manager.cc:194] Re-registered known tserver with Master: 93dbcfa6e0784096bed84ddb3f611598 (127.27.180.131:39783)
I20260811 18:55:31.328128 7768 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.27.180.131@KRBTEST.COM'} at 127.27.180.131:57505
I20260811 18:55:31.332993 10150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:31.317889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44460 (local address 127.27.180.189:45319)
0811 18:55:31.319398 (+ 1509us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:31.319402 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:31.319415 (+ 13us) server_negotiation.cc:407] Connection header received
0811 18:55:31.319449 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:31.319451 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:31.319501 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:31.319583 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:31.322306 (+ 2723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:31.323483 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:31.326917 (+ 3434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:31.327241 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:31.327624 (+ 383us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:31.329121 (+ 1497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:31.329126 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:31.329129 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:31.329155 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:31.331148 (+ 1993us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:31.331498 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:31.331502 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:31.331504 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:31.331594 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:31.331884 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:31.331888 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:31.331890 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:31.332100 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:31.332237 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:31.332709 (+ 472us) server_negotiation.cc:299] Negotiation successful
0811 18:55:31.332835 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1432,"thread_start_us":91,"threads_started":1}
I20260811 18:55:31.334404 10156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:31.318718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36150 (local address 127.27.180.190:46847)
0811 18:55:31.318971 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:31.318976 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:31.321434 (+ 2458us) server_negotiation.cc:407] Connection header received
0811 18:55:31.322672 (+ 1238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:31.322675 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:31.322722 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:31.322809 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:31.324297 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:31.325008 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:31.328658 (+ 3650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:31.329001 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:31.329116 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:31.330538 (+ 1422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:31.330541 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:31.330544 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:31.330568 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:31.332289 (+ 1721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:31.332760 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:31.332763 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:31.332765 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:31.332852 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:31.333154 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:31.333157 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:31.333159 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:31.333379 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:31.333486 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:31.334134 (+ 648us) server_negotiation.cc:299] Negotiation successful
0811 18:55:31.334252 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":73,"threads_started":1}
I20260811 18:55:31.334578 10155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:31.318331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54700 (local address 127.27.180.188:34747)
0811 18:55:31.318583 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:31.318587 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:31.319934 (+ 1347us) server_negotiation.cc:407] Connection header received
0811 18:55:31.321186 (+ 1252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:31.321190 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:31.321244 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:31.321320 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:31.325155 (+ 3835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:31.325824 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:31.329165 (+ 3341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:31.329524 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:31.329601 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:31.331097 (+ 1496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:31.331101 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:31.331104 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:31.331131 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:31.332701 (+ 1570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:31.333395 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:31.333399 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:31.333400 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:31.333484 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:31.333748 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:31.333752 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:31.333753 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:31.333955 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:31.334054 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:31.334300 (+ 246us) server_negotiation.cc:299] Negotiation successful
0811 18:55:31.334430 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":85,"threads_started":1}
I20260811 18:55:31.336833 7768 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36150:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:55:31.339864 7768 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:55:31.342320 8859 sys_catalog.cc:455] T 00000000000000000000000000000000 P fec531dd027244268997c383dda94895 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "fec531dd027244268997c383dda94895" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "fec531dd027244268997c383dda94895" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 46847 } } peers { permanent_uuid: "e3781302d19a4a8a9342e98ea95ddc93" member_type: VOTER last_known_addr { host: "127.27.180.189" port: 45319 } attrs { promote: false } } peers { permanent_uuid: "9149de133657420baaadd406d6f504fd" member_type: NON_VOTER last_known_addr { host: "127.27.180.188" port: 34747 } attrs { promote: true } } }
I20260811 18:55:31.342463 8859 sys_catalog.cc:458] T 00000000000000000000000000000000 P fec531dd027244268997c383dda94895 [sys.catalog]: This master's current role is: LEADER
I20260811 18:55:31.345577 10168 raft_consensus.cc:1050] T 00000000000000000000000000000000 P fec531dd027244268997c383dda94895: attempting to promote NON_VOTER 9149de133657420baaadd406d6f504fd to VOTER
I20260811 18:55:31.345940 10168 consensus_queue.cc:237] T 00000000000000000000000000000000 P fec531dd027244268997c383dda94895 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "fec531dd027244268997c383dda94895" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 46847 } } peers { permanent_uuid: "e3781302d19a4a8a9342e98ea95ddc93" member_type: VOTER last_known_addr { host: "127.27.180.189" port: 45319 } attrs { promote: false } } peers { permanent_uuid: "9149de133657420baaadd406d6f504fd" member_type: VOTER last_known_addr { host: "127.27.180.188" port: 34747 } attrs { promote: false } }
I20260811 18:55:31.348759 8956 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e3781302d19a4a8a9342e98ea95ddc93 [term 1 FOLLOWER]: Refusing update from remote peer fec531dd027244268997c383dda94895: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260811 18:55:31.349146 9316 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9149de133657420baaadd406d6f504fd [term 1 LEARNER]: Refusing update from remote peer fec531dd027244268997c383dda94895: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260811 18:55:31.349174 10160 consensus_queue.cc:1035] T 00000000000000000000000000000000 P fec531dd027244268997c383dda94895 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3781302d19a4a8a9342e98ea95ddc93" member_type: VOTER last_known_addr { host: "127.27.180.189" port: 45319 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260811 18:55:31.349498 10160 consensus_queue.cc:1035] T 00000000000000000000000000000000 P fec531dd027244268997c383dda94895 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9149de133657420baaadd406d6f504fd" member_type: VOTER last_known_addr { host: "127.27.180.188" port: 34747 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260811 18:55:31.350838 8859 raft_consensus.cc:2932] T 00000000000000000000000000000000 P fec531dd027244268997c383dda94895 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 9149de133657420baaadd406d6f504fd (127.27.180.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "fec531dd027244268997c383dda94895" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 46847 } } peers { permanent_uuid: "e3781302d19a4a8a9342e98ea95ddc93" member_type: VOTER last_known_addr { host: "127.27.180.189" port: 45319 } attrs { promote: false } } peers { permanent_uuid: "9149de133657420baaadd406d6f504fd" member_type: VOTER last_known_addr { host: "127.27.180.188" port: 34747 } attrs { promote: false } } }
I20260811 18:55:31.351361 8956 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e3781302d19a4a8a9342e98ea95ddc93 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 9149de133657420baaadd406d6f504fd (127.27.180.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "fec531dd027244268997c383dda94895" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 46847 } } peers { permanent_uuid: "e3781302d19a4a8a9342e98ea95ddc93" member_type: VOTER last_known_addr { host: "127.27.180.189" port: 45319 } attrs { promote: false } } peers { permanent_uuid: "9149de133657420baaadd406d6f504fd" member_type: VOTER last_known_addr { host: "127.27.180.188" port: 34747 } attrs { promote: false } } }
I20260811 18:55:31.351780 9316 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9149de133657420baaadd406d6f504fd [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 9149de133657420baaadd406d6f504fd (127.27.180.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "fec531dd027244268997c383dda94895" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 46847 } } peers { permanent_uuid: "e3781302d19a4a8a9342e98ea95ddc93" member_type: VOTER last_known_addr { host: "127.27.180.189" port: 45319 } attrs { promote: false } } peers { permanent_uuid: "9149de133657420baaadd406d6f504fd" member_type: VOTER last_known_addr { host: "127.27.180.188" port: 34747 } attrs { promote: false } } }
I20260811 18:55:31.352325 8859 sys_catalog.cc:455] T 00000000000000000000000000000000 P fec531dd027244268997c383dda94895 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "fec531dd027244268997c383dda94895" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "fec531dd027244268997c383dda94895" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 46847 } } peers { permanent_uuid: "e3781302d19a4a8a9342e98ea95ddc93" member_type: VOTER last_known_addr { host: "127.27.180.189" port: 45319 } attrs { promote: false } } peers { permanent_uuid: "9149de133657420baaadd406d6f504fd" member_type: VOTER last_known_addr { host: "127.27.180.188" port: 34747 } attrs { promote: false } } }
I20260811 18:55:31.352437 8859 sys_catalog.cc:458] T 00000000000000000000000000000000 P fec531dd027244268997c383dda94895 [sys.catalog]: This master's current role is: LEADER
I20260811 18:55:31.352546 8859 sys_catalog.cc:455] T 00000000000000000000000000000000 P fec531dd027244268997c383dda94895 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "fec531dd027244268997c383dda94895" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "fec531dd027244268997c383dda94895" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 46847 } } peers { permanent_uuid: "e3781302d19a4a8a9342e98ea95ddc93" member_type: VOTER last_known_addr { host: "127.27.180.189" port: 45319 } attrs { promote: false } } peers { permanent_uuid: "9149de133657420baaadd406d6f504fd" member_type: VOTER last_known_addr { host: "127.27.180.188" port: 34747 } attrs { promote: false } } }
I20260811 18:55:31.352631 8859 sys_catalog.cc:458] T 00000000000000000000000000000000 P fec531dd027244268997c383dda94895 [sys.catalog]: This master's current role is: LEADER
I20260811 18:55:31.352732 8859 sys_catalog.cc:455] T 00000000000000000000000000000000 P fec531dd027244268997c383dda94895 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "fec531dd027244268997c383dda94895" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "fec531dd027244268997c383dda94895" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 46847 } } peers { permanent_uuid: "e3781302d19a4a8a9342e98ea95ddc93" member_type: VOTER last_known_addr { host: "127.27.180.189" port: 45319 } attrs { promote: false } } peers { permanent_uuid: "9149de133657420baaadd406d6f504fd" member_type: VOTER last_known_addr { host: "127.27.180.188" port: 34747 } attrs { promote: false } } }
I20260811 18:55:31.352820 8859 sys_catalog.cc:458] T 00000000000000000000000000000000 P fec531dd027244268997c383dda94895 [sys.catalog]: This master's current role is: LEADER
I20260811 18:55:31.353511 9370 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9149de133657420baaadd406d6f504fd [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "fec531dd027244268997c383dda94895" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "fec531dd027244268997c383dda94895" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 46847 } } peers { permanent_uuid: "e3781302d19a4a8a9342e98ea95ddc93" member_type: VOTER last_known_addr { host: "127.27.180.189" port: 45319 } attrs { promote: false } } peers { permanent_uuid: "9149de133657420baaadd406d6f504fd" member_type: VOTER last_known_addr { host: "127.27.180.188" port: 34747 } attrs { promote: false } } }
I20260811 18:55:31.353619 9370 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9149de133657420baaadd406d6f504fd [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:55:31.353698 10181 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3781302d19a4a8a9342e98ea95ddc93 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "fec531dd027244268997c383dda94895" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "fec531dd027244268997c383dda94895" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 46847 } } peers { permanent_uuid: "e3781302d19a4a8a9342e98ea95ddc93" member_type: VOTER last_known_addr { host: "127.27.180.189" port: 45319 } attrs { promote: false } } peers { permanent_uuid: "9149de133657420baaadd406d6f504fd" member_type: VOTER last_known_addr { host: "127.27.180.188" port: 34747 } attrs { promote: false } } }
I20260811 18:55:31.353821 10181 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3781302d19a4a8a9342e98ea95ddc93 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:55:31.359457 9880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:31.354574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.131:39783 (local address 127.0.0.1:39916)
0811 18:55:31.354683 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:31.354698 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:31.354767 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:31.355707 (+ 940us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:31.355710 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:31.355730 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:55:31.355823 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:31.355830 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:31.356829 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:31.356832 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:31.358002 (+ 1170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:31.358013 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:31.359090 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:31.359094 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:31.359179 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:31.359204 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 18:55:31.359251 (+ 47us) client_negotiation.cc:240] Negotiation successful
0811 18:55:31.359321 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260811 18:55:31.360158 10182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:31.354748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39916 (local address 127.27.180.131:39783)
0811 18:55:31.355049 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:31.355062 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:31.355085 (+ 23us) server_negotiation.cc:407] Connection header received
0811 18:55:31.355164 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:31.355170 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:31.355532 (+ 362us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:31.355740 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:55:31.355920 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:31.356721 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:31.358130 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:31.358957 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:31.359731 (+ 774us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:31.359869 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:31.359921 (+ 52us) server_negotiation.cc:299] Negotiation successful
0811 18:55:31.360015 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":104,"threads_started":1}
I20260811 18:55:31.360525 10183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:31.355070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.129:32985 (local address 127.0.0.1:59056)
0811 18:55:31.355281 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:31.355294 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:31.355372 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:31.356581 (+ 1209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:31.356584 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:31.356604 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:55:31.356703 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:31.356709 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:31.357835 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:31.357838 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:31.358948 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:31.358957 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:31.360128 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:31.360132 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:31.360225 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:31.360250 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 18:55:31.360324 (+ 74us) client_negotiation.cc:240] Negotiation successful
0811 18:55:31.360392 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"mutex_wait_us":43,"thread_start_us":85,"threads_started":1}
I20260811 18:55:31.360579 10186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:31.355244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59056 (local address 127.27.180.129:32985)
0811 18:55:31.356222 (+ 978us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:31.356233 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:31.356259 (+ 26us) server_negotiation.cc:407] Connection header received
0811 18:55:31.356317 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:31.356321 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:31.356456 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:31.356568 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:55:31.356839 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:31.357741 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:31.359083 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:31.359832 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:31.359954 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:31.360064 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:31.360356 (+ 292us) server_negotiation.cc:299] Negotiation successful
0811 18:55:31.360455 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":898,"thread_start_us":67,"threads_started":1}
I20260811 18:55:31.360880 10187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:31.355358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.130:45897 (local address 127.0.0.1:58908)
0811 18:55:31.356748 (+ 1390us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:31.356760 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:31.356844 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:31.357348 (+ 504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:31.357352 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:31.357372 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:55:31.357454 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:31.357459 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:31.358530 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:31.358533 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:31.359568 (+ 1035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:31.359579 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:31.360496 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:31.360499 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:31.360582 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:31.360603 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 18:55:31.360677 (+ 74us) client_negotiation.cc:240] Negotiation successful
0811 18:55:31.360739 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1337,"mutex_wait_us":24,"thread_start_us":70,"threads_started":1}
I20260811 18:55:31.361302 10188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:31.355676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58908 (local address 127.27.180.130:45897)
0811 18:55:31.356064 (+ 388us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:31.356075 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:31.356929 (+ 854us) server_negotiation.cc:407] Connection header received
0811 18:55:31.357002 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:31.357007 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:31.357199 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:31.357352 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:55:31.357560 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:31.358436 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:31.359696 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:31.360196 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:31.360308 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:31.360431 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:31.361042 (+ 611us) server_negotiation.cc:299] Negotiation successful
0811 18:55:31.361168 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":91,"threads_started":1}
I20260811 18:55:31.363133 9999 tablet_service.cc:1467] Processing CreateTablet for tablet 914a4a144b624c31b6d439067d8173e2 (DEFAULT_TABLE table=test-table [id=cffc3a066c0742cabaa610becd0ef9a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:55:31.363651 9755 tablet_service.cc:1467] Processing CreateTablet for tablet 914a4a144b624c31b6d439067d8173e2 (DEFAULT_TABLE table=test-table [id=cffc3a066c0742cabaa610becd0ef9a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:55:31.363590 9500 tablet_service.cc:1467] Processing CreateTablet for tablet 914a4a144b624c31b6d439067d8173e2 (DEFAULT_TABLE table=test-table [id=cffc3a066c0742cabaa610becd0ef9a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:55:31.364276 9999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 914a4a144b624c31b6d439067d8173e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:55:31.364503 9755 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 914a4a144b624c31b6d439067d8173e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:55:31.364557 9500 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 914a4a144b624c31b6d439067d8173e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:55:31.371111 10189 tablet_bootstrap.cc:492] T 914a4a144b624c31b6d439067d8173e2 P 485e9ca42b2a45aa9aeeb564daf0edda: Bootstrap starting.
I20260811 18:55:31.372740 10192 tablet_bootstrap.cc:492] T 914a4a144b624c31b6d439067d8173e2 P b1221343a5f8419ab3f58e8e328cc30c: Bootstrap starting.
I20260811 18:55:31.373427 10195 tablet_bootstrap.cc:492] T 914a4a144b624c31b6d439067d8173e2 P 93dbcfa6e0784096bed84ddb3f611598: Bootstrap starting.
I20260811 18:55:31.374076 10189 tablet_bootstrap.cc:654] T 914a4a144b624c31b6d439067d8173e2 P 485e9ca42b2a45aa9aeeb564daf0edda: Neither blocks nor log segments found. Creating new log.
I20260811 18:55:31.374629 10192 tablet_bootstrap.cc:654] T 914a4a144b624c31b6d439067d8173e2 P b1221343a5f8419ab3f58e8e328cc30c: Neither blocks nor log segments found. Creating new log.
I20260811 18:55:31.374948 10189 log.cc:824] T 914a4a144b624c31b6d439067d8173e2 P 485e9ca42b2a45aa9aeeb564daf0edda: Log is configured to *not* fsync() on all Append() calls
I20260811 18:55:31.375313 10192 log.cc:824] T 914a4a144b624c31b6d439067d8173e2 P b1221343a5f8419ab3f58e8e328cc30c: Log is configured to *not* fsync() on all Append() calls
I20260811 18:55:31.375535 10195 tablet_bootstrap.cc:654] T 914a4a144b624c31b6d439067d8173e2 P 93dbcfa6e0784096bed84ddb3f611598: Neither blocks nor log segments found. Creating new log.
I20260811 18:55:31.376324 10195 log.cc:824] T 914a4a144b624c31b6d439067d8173e2 P 93dbcfa6e0784096bed84ddb3f611598: Log is configured to *not* fsync() on all Append() calls
I20260811 18:55:31.377050 10189 tablet_bootstrap.cc:492] T 914a4a144b624c31b6d439067d8173e2 P 485e9ca42b2a45aa9aeeb564daf0edda: No bootstrap required, opened a new log
I20260811 18:55:31.377192 10192 tablet_bootstrap.cc:492] T 914a4a144b624c31b6d439067d8173e2 P b1221343a5f8419ab3f58e8e328cc30c: No bootstrap required, opened a new log
I20260811 18:55:31.377272 10189 ts_tablet_manager.cc:1397] T 914a4a144b624c31b6d439067d8173e2 P 485e9ca42b2a45aa9aeeb564daf0edda: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.001s
I20260811 18:55:31.377341 10192 ts_tablet_manager.cc:1397] T 914a4a144b624c31b6d439067d8173e2 P b1221343a5f8419ab3f58e8e328cc30c: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 18:55:31.378580 10195 tablet_bootstrap.cc:492] T 914a4a144b624c31b6d439067d8173e2 P 93dbcfa6e0784096bed84ddb3f611598: No bootstrap required, opened a new log
I20260811 18:55:31.378762 10195 ts_tablet_manager.cc:1397] T 914a4a144b624c31b6d439067d8173e2 P 93dbcfa6e0784096bed84ddb3f611598: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260811 18:55:31.381866 10192 raft_consensus.cc:348] T 914a4a144b624c31b6d439067d8173e2 P b1221343a5f8419ab3f58e8e328cc30c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93dbcfa6e0784096bed84ddb3f611598" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 39783 } } peers { permanent_uuid: "b1221343a5f8419ab3f58e8e328cc30c" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 32985 } } peers { permanent_uuid: "485e9ca42b2a45aa9aeeb564daf0edda" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 45897 } }
I20260811 18:55:31.382149 10192 raft_consensus.cc:374] T 914a4a144b624c31b6d439067d8173e2 P b1221343a5f8419ab3f58e8e328cc30c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:55:31.382032 10189 raft_consensus.cc:348] T 914a4a144b624c31b6d439067d8173e2 P 485e9ca42b2a45aa9aeeb564daf0edda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93dbcfa6e0784096bed84ddb3f611598" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 39783 } } peers { permanent_uuid: "b1221343a5f8419ab3f58e8e328cc30c" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 32985 } } peers { permanent_uuid: "485e9ca42b2a45aa9aeeb564daf0edda" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 45897 } }
I20260811 18:55:31.382215 10192 raft_consensus.cc:729] T 914a4a144b624c31b6d439067d8173e2 P b1221343a5f8419ab3f58e8e328cc30c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1221343a5f8419ab3f58e8e328cc30c, State: Initialized, Role: FOLLOWER
I20260811 18:55:31.382237 10189 raft_consensus.cc:374] T 914a4a144b624c31b6d439067d8173e2 P 485e9ca42b2a45aa9aeeb564daf0edda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:55:31.382285 10189 raft_consensus.cc:729] T 914a4a144b624c31b6d439067d8173e2 P 485e9ca42b2a45aa9aeeb564daf0edda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 485e9ca42b2a45aa9aeeb564daf0edda, State: Initialized, Role: FOLLOWER
I20260811 18:55:31.382251 10195 raft_consensus.cc:348] T 914a4a144b624c31b6d439067d8173e2 P 93dbcfa6e0784096bed84ddb3f611598 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93dbcfa6e0784096bed84ddb3f611598" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 39783 } } peers { permanent_uuid: "b1221343a5f8419ab3f58e8e328cc30c" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 32985 } } peers { permanent_uuid: "485e9ca42b2a45aa9aeeb564daf0edda" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 45897 } }
I20260811 18:55:31.382534 10195 raft_consensus.cc:374] T 914a4a144b624c31b6d439067d8173e2 P 93dbcfa6e0784096bed84ddb3f611598 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:55:31.382591 10195 raft_consensus.cc:729] T 914a4a144b624c31b6d439067d8173e2 P 93dbcfa6e0784096bed84ddb3f611598 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93dbcfa6e0784096bed84ddb3f611598, State: Initialized, Role: FOLLOWER
I20260811 18:55:31.382738 10192 consensus_queue.cc:260] T 914a4a144b624c31b6d439067d8173e2 P b1221343a5f8419ab3f58e8e328cc30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93dbcfa6e0784096bed84ddb3f611598" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 39783 } } peers { permanent_uuid: "b1221343a5f8419ab3f58e8e328cc30c" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 32985 } } peers { permanent_uuid: "485e9ca42b2a45aa9aeeb564daf0edda" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 45897 } }
I20260811 18:55:31.382738 10189 consensus_queue.cc:260] T 914a4a144b624c31b6d439067d8173e2 P 485e9ca42b2a45aa9aeeb564daf0edda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93dbcfa6e0784096bed84ddb3f611598" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 39783 } } peers { permanent_uuid: "b1221343a5f8419ab3f58e8e328cc30c" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 32985 } } peers { permanent_uuid: "485e9ca42b2a45aa9aeeb564daf0edda" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 45897 } }
I20260811 18:55:31.383047 10195 consensus_queue.cc:260] T 914a4a144b624c31b6d439067d8173e2 P 93dbcfa6e0784096bed84ddb3f611598 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93dbcfa6e0784096bed84ddb3f611598" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 39783 } } peers { permanent_uuid: "b1221343a5f8419ab3f58e8e328cc30c" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 32985 } } peers { permanent_uuid: "485e9ca42b2a45aa9aeeb564daf0edda" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 45897 } }
I20260811 18:55:31.383771 10189 ts_tablet_manager.cc:1428] T 914a4a144b624c31b6d439067d8173e2 P 485e9ca42b2a45aa9aeeb564daf0edda: Time spent starting tablet: real 0.006s user 0.000s sys 0.006s
I20260811 18:55:31.383771 10192 ts_tablet_manager.cc:1428] T 914a4a144b624c31b6d439067d8173e2 P b1221343a5f8419ab3f58e8e328cc30c: Time spent starting tablet: real 0.006s user 0.003s sys 0.004s
I20260811 18:55:31.384016 10195 ts_tablet_manager.cc:1428] T 914a4a144b624c31b6d439067d8173e2 P 93dbcfa6e0784096bed84ddb3f611598: Time spent starting tablet: real 0.005s user 0.000s sys 0.006s
I20260811 18:55:31.384034 9867 heartbeater.cc:503] Master 127.27.180.190:46847 was elected leader, sending a full tablet report...
I20260811 18:55:31.385438 10132 heartbeater.cc:503] Master 127.27.180.190:46847 was elected leader, sending a full tablet report...
I20260811 18:55:31.387785 9634 heartbeater.cc:503] Master 127.27.180.190:46847 was elected leader, sending a full tablet report...
I20260811 18:55:31.460636 10206 raft_consensus.cc:482] T 914a4a144b624c31b6d439067d8173e2 P 485e9ca42b2a45aa9aeeb564daf0edda [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:55:31.460834 10206 raft_consensus.cc:504] T 914a4a144b624c31b6d439067d8173e2 P 485e9ca42b2a45aa9aeeb564daf0edda [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93dbcfa6e0784096bed84ddb3f611598" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 39783 } } peers { permanent_uuid: "b1221343a5f8419ab3f58e8e328cc30c" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 32985 } } peers { permanent_uuid: "485e9ca42b2a45aa9aeeb564daf0edda" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 45897 } }
I20260811 18:55:31.461990 10206 leader_election.cc:288] T 914a4a144b624c31b6d439067d8173e2 P 485e9ca42b2a45aa9aeeb564daf0edda [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 93dbcfa6e0784096bed84ddb3f611598 (127.27.180.131:39783), b1221343a5f8419ab3f58e8e328cc30c (127.27.180.129:32985)
I20260811 18:55:31.465621 10186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:31.462379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.130:52089 (local address 127.27.180.129:32985)
0811 18:55:31.462512 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:31.462516 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:31.462531 (+ 15us) server_negotiation.cc:407] Connection header received
0811 18:55:31.462654 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:31.462657 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:31.462700 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:31.462763 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:55:31.463084 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:31.463606 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:31.464659 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:31.465042 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:31.465184 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:31.465242 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:31.465419 (+ 177us) server_negotiation.cc:299] Negotiation successful
0811 18:55:31.465492 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 18:55:31.465621 10182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:31.462377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.130:46185 (local address 127.27.180.131:39783)
0811 18:55:31.462498 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:31.462502 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:31.462646 (+ 144us) server_negotiation.cc:407] Connection header received
0811 18:55:31.462771 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:31.462774 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:31.462813 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:31.462875 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:55:31.463146 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:31.463631 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:31.464663 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:31.465045 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:31.465188 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:31.465242 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:31.465425 (+ 183us) server_negotiation.cc:299] Negotiation successful
0811 18:55:31.465495 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 18:55:31.465629 9872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:31.462271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.131:39783 (local address 127.27.180.130:46185)
0811 18:55:31.462566 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:31.462577 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:31.462667 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:31.462895 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:31.462898 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:31.462913 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:55:31.463020 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:31.463025 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:31.463737 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:31.463740 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:31.464543 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:31.464552 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:31.465188 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:31.465191 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:31.465317 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:31.465334 (+ 17us) client_negotiation.cc:769] Sending connection context
0811 18:55:31.465399 (+ 65us) client_negotiation.cc:240] Negotiation successful
0811 18:55:31.465474 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"mutex_wait_us":169}
I20260811 18:55:31.465639 9875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:31.462267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.129:32985 (local address 127.27.180.130:52089)
0811 18:55:31.462403 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:31.462420 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:31.462544 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:31.462791 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:31.462794 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:31.462809 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:55:31.462938 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:31.462944 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:31.463716 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:31.463719 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:31.464540 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:31.464551 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:31.465184 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:31.465187 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:31.465297 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:31.465322 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 18:55:31.465387 (+ 65us) client_negotiation.cc:240] Negotiation successful
0811 18:55:31.465488 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42,"mutex_wait_us":8}
I20260811 18:55:31.466359 9535 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "914a4a144b624c31b6d439067d8173e2" candidate_uuid: "485e9ca42b2a45aa9aeeb564daf0edda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1221343a5f8419ab3f58e8e328cc30c" is_pre_election: true
I20260811 18:55:31.466358 10032 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "914a4a144b624c31b6d439067d8173e2" candidate_uuid: "485e9ca42b2a45aa9aeeb564daf0edda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93dbcfa6e0784096bed84ddb3f611598" is_pre_election: true
I20260811 18:55:31.466653 10032 raft_consensus.cc:2454] T 914a4a144b624c31b6d439067d8173e2 P 93dbcfa6e0784096bed84ddb3f611598 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 485e9ca42b2a45aa9aeeb564daf0edda in term 0.
I20260811 18:55:31.466653 9535 raft_consensus.cc:2454] T 914a4a144b624c31b6d439067d8173e2 P b1221343a5f8419ab3f58e8e328cc30c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 485e9ca42b2a45aa9aeeb564daf0edda in term 0.
I20260811 18:55:31.467101 9679 leader_election.cc:302] T 914a4a144b624c31b6d439067d8173e2 P 485e9ca42b2a45aa9aeeb564daf0edda [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 485e9ca42b2a45aa9aeeb564daf0edda, 93dbcfa6e0784096bed84ddb3f611598; no voters:
I20260811 18:55:31.467370 10206 raft_consensus.cc:2781] T 914a4a144b624c31b6d439067d8173e2 P 485e9ca42b2a45aa9aeeb564daf0edda [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:55:31.467463 10206 raft_consensus.cc:482] T 914a4a144b624c31b6d439067d8173e2 P 485e9ca42b2a45aa9aeeb564daf0edda [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:55:31.467514 10206 raft_consensus.cc:3037] T 914a4a144b624c31b6d439067d8173e2 P 485e9ca42b2a45aa9aeeb564daf0edda [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:55:31.468557 10206 raft_consensus.cc:504] T 914a4a144b624c31b6d439067d8173e2 P 485e9ca42b2a45aa9aeeb564daf0edda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93dbcfa6e0784096bed84ddb3f611598" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 39783 } } peers { permanent_uuid: "b1221343a5f8419ab3f58e8e328cc30c" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 32985 } } peers { permanent_uuid: "485e9ca42b2a45aa9aeeb564daf0edda" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 45897 } }
I20260811 18:55:31.468900 10206 leader_election.cc:288] T 914a4a144b624c31b6d439067d8173e2 P 485e9ca42b2a45aa9aeeb564daf0edda [CANDIDATE]: Term 1 election: Requested vote from peers 93dbcfa6e0784096bed84ddb3f611598 (127.27.180.131:39783), b1221343a5f8419ab3f58e8e328cc30c (127.27.180.129:32985)
I20260811 18:55:31.469273 9535 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "914a4a144b624c31b6d439067d8173e2" candidate_uuid: "485e9ca42b2a45aa9aeeb564daf0edda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1221343a5f8419ab3f58e8e328cc30c"
I20260811 18:55:31.469280 10032 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "914a4a144b624c31b6d439067d8173e2" candidate_uuid: "485e9ca42b2a45aa9aeeb564daf0edda" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93dbcfa6e0784096bed84ddb3f611598"
I20260811 18:55:31.469380 9535 raft_consensus.cc:3037] T 914a4a144b624c31b6d439067d8173e2 P b1221343a5f8419ab3f58e8e328cc30c [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:55:31.469388 10032 raft_consensus.cc:3037] T 914a4a144b624c31b6d439067d8173e2 P 93dbcfa6e0784096bed84ddb3f611598 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:55:31.470355 9535 raft_consensus.cc:2454] T 914a4a144b624c31b6d439067d8173e2 P b1221343a5f8419ab3f58e8e328cc30c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 485e9ca42b2a45aa9aeeb564daf0edda in term 1.
I20260811 18:55:31.470633 10032 raft_consensus.cc:2454] T 914a4a144b624c31b6d439067d8173e2 P 93dbcfa6e0784096bed84ddb3f611598 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 485e9ca42b2a45aa9aeeb564daf0edda in term 1.
I20260811 18:55:31.470638 9681 leader_election.cc:302] T 914a4a144b624c31b6d439067d8173e2 P 485e9ca42b2a45aa9aeeb564daf0edda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 485e9ca42b2a45aa9aeeb564daf0edda, b1221343a5f8419ab3f58e8e328cc30c; no voters:
I20260811 18:55:31.470819 10206 raft_consensus.cc:2781] T 914a4a144b624c31b6d439067d8173e2 P 485e9ca42b2a45aa9aeeb564daf0edda [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:55:31.471056 10206 raft_consensus.cc:686] T 914a4a144b624c31b6d439067d8173e2 P 485e9ca42b2a45aa9aeeb564daf0edda [term 1 LEADER]: Becoming Leader. State: Replica: 485e9ca42b2a45aa9aeeb564daf0edda, State: Running, Role: LEADER
I20260811 18:55:31.471382 10206 consensus_queue.cc:237] T 914a4a144b624c31b6d439067d8173e2 P 485e9ca42b2a45aa9aeeb564daf0edda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93dbcfa6e0784096bed84ddb3f611598" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 39783 } } peers { permanent_uuid: "b1221343a5f8419ab3f58e8e328cc30c" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 32985 } } peers { permanent_uuid: "485e9ca42b2a45aa9aeeb564daf0edda" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 45897 } }
I20260811 18:55:31.474663 7767 catalog_manager.cc:5495] T 914a4a144b624c31b6d439067d8173e2 P 485e9ca42b2a45aa9aeeb564daf0edda reported cstate change: term changed from 0 to 1, leader changed from <none> to 485e9ca42b2a45aa9aeeb564daf0edda (127.27.180.130). New cstate: current_term: 1 leader_uuid: "485e9ca42b2a45aa9aeeb564daf0edda" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93dbcfa6e0784096bed84ddb3f611598" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 39783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1221343a5f8419ab3f58e8e328cc30c" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 32985 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "485e9ca42b2a45aa9aeeb564daf0edda" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 45897 } health_report { overall_health: HEALTHY } } }
I20260811 18:55:31.508767 10188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:31.505368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58916 (local address 127.27.180.130:45897)
0811 18:55:31.505495 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:31.505500 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:31.505514 (+ 14us) server_negotiation.cc:407] Connection header received
0811 18:55:31.505619 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:31.505621 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:31.505677 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:31.505766 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:55:31.506051 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:31.506550 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:31.507382 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:31.507596 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:31.507684 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:31.507992 (+ 308us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:55:31.508393 (+ 401us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:55:31.508502 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:31.508574 (+ 72us) server_negotiation.cc:299] Negotiation successful
0811 18:55:31.508630 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 18:55:31.514252 10032 raft_consensus.cc:1261] T 914a4a144b624c31b6d439067d8173e2 P 93dbcfa6e0784096bed84ddb3f611598 [term 1 FOLLOWER]: Refusing update from remote peer 485e9ca42b2a45aa9aeeb564daf0edda: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:55:31.514253 9535 raft_consensus.cc:1261] T 914a4a144b624c31b6d439067d8173e2 P b1221343a5f8419ab3f58e8e328cc30c [term 1 FOLLOWER]: Refusing update from remote peer 485e9ca42b2a45aa9aeeb564daf0edda: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:55:31.514928 10206 consensus_queue.cc:1035] T 914a4a144b624c31b6d439067d8173e2 P 485e9ca42b2a45aa9aeeb564daf0edda [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1221343a5f8419ab3f58e8e328cc30c" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 32985 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:55:31.515158 10210 consensus_queue.cc:1035] T 914a4a144b624c31b6d439067d8173e2 P 485e9ca42b2a45aa9aeeb564daf0edda [LEADER]: Connected to new peer: Peer: permanent_uuid: "93dbcfa6e0784096bed84ddb3f611598" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 39783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:55:31.525861 10217 mvcc.cc:205] Tried to move back new op lower bound from 7317399681075597312 to 7317399680909582336. Current Snapshot: MvccSnapshot[applied={T|T < 7317399681075597312}]
I20260811 18:55:31.531414 10218 mvcc.cc:205] Tried to move back new op lower bound from 7317399681075597312 to 7317399680909582336. Current Snapshot: MvccSnapshot[applied={T|T < 7317399681075597312 or (T in {7317399681075597312})}]
I20260811 18:55:31.532050 7768 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36150:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:55:31.532219 7768 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36150:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:55:31.536259 7768 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fec531dd027244268997c383dda94895: Sending DeleteTablet for 3 replicas of tablet 914a4a144b624c31b6d439067d8173e2
I20260811 18:55:31.537027 9999 tablet_service.cc:1514] Processing DeleteTablet for tablet 914a4a144b624c31b6d439067d8173e2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:55:31 UTC) from {username='oryx', principal='oryx/127.27.180.190@KRBTEST.COM'} at 127.0.0.1:39916
I20260811 18:55:31.537559 10234 tablet_replica.cc:330] T 914a4a144b624c31b6d439067d8173e2 P 93dbcfa6e0784096bed84ddb3f611598: stopping tablet replica
I20260811 18:55:31.537938 10234 raft_consensus.cc:2229] T 914a4a144b624c31b6d439067d8173e2 P 93dbcfa6e0784096bed84ddb3f611598 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:55:31.538246 10234 raft_consensus.cc:2258] T 914a4a144b624c31b6d439067d8173e2 P 93dbcfa6e0784096bed84ddb3f611598 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:55:31.538215 9500 tablet_service.cc:1514] Processing DeleteTablet for tablet 914a4a144b624c31b6d439067d8173e2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:55:31 UTC) from {username='oryx', principal='oryx/127.27.180.190@KRBTEST.COM'} at 127.0.0.1:59056
I20260811 18:55:31.538890 9755 tablet_service.cc:1514] Processing DeleteTablet for tablet 914a4a144b624c31b6d439067d8173e2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:55:31 UTC) from {username='oryx', principal='oryx/127.27.180.190@KRBTEST.COM'} at 127.0.0.1:58908
I20260811 18:55:31.539510 10235 tablet_replica.cc:330] T 914a4a144b624c31b6d439067d8173e2 P b1221343a5f8419ab3f58e8e328cc30c: stopping tablet replica
I20260811 18:55:31.539947 10235 raft_consensus.cc:2229] T 914a4a144b624c31b6d439067d8173e2 P b1221343a5f8419ab3f58e8e328cc30c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:55:31.540207 10234 ts_tablet_manager.cc:1905] T 914a4a144b624c31b6d439067d8173e2 P 93dbcfa6e0784096bed84ddb3f611598: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:55:31.540372 10235 raft_consensus.cc:2258] T 914a4a144b624c31b6d439067d8173e2 P b1221343a5f8419ab3f58e8e328cc30c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:55:31.542002 10235 ts_tablet_manager.cc:1905] T 914a4a144b624c31b6d439067d8173e2 P b1221343a5f8419ab3f58e8e328cc30c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:55:31.543010 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 7699
I20260811 18:55:31.553181 10234 ts_tablet_manager.cc:1918] T 914a4a144b624c31b6d439067d8173e2 P 93dbcfa6e0784096bed84ddb3f611598: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:55:31.553418 10234 log.cc:1196] T 914a4a144b624c31b6d439067d8173e2 P 93dbcfa6e0784096bed84ddb3f611598: Deleting WAL directory at /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/ts-2/wal/wals/914a4a144b624c31b6d439067d8173e2
I20260811 18:55:31.553964 10234 ts_tablet_manager.cc:1939] T 914a4a144b624c31b6d439067d8173e2 P 93dbcfa6e0784096bed84ddb3f611598: Deleting consensus metadata
I20260811 18:55:31.556368 10235 ts_tablet_manager.cc:1918] T 914a4a144b624c31b6d439067d8173e2 P b1221343a5f8419ab3f58e8e328cc30c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:55:31.556458 10235 log.cc:1196] T 914a4a144b624c31b6d439067d8173e2 P b1221343a5f8419ab3f58e8e328cc30c: Deleting WAL directory at /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/ts-0/wal/wals/914a4a144b624c31b6d439067d8173e2
I20260811 18:55:31.556685 10235 ts_tablet_manager.cc:1939] T 914a4a144b624c31b6d439067d8173e2 P b1221343a5f8419ab3f58e8e328cc30c: Deleting consensus metadata
I20260811 18:55:31.565034 10150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:31.542064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44476 (local address 127.27.180.189:45319)
0811 18:55:31.542207 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:31.542212 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:31.555991 (+ 13779us) server_negotiation.cc:407] Connection header received
0811 18:55:31.556250 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:31.556255 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:31.556321 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:31.556402 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:31.557957 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:31.558608 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:31.559980 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:31.560242 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:31.560318 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:31.562086 (+ 1768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:31.562089 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:31.562092 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:31.562117 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:31.563458 (+ 1341us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:31.563853 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:31.563855 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:31.563857 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:31.563956 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:31.564211 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:31.564215 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:31.564217 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:31.564396 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:31.564515 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:31.564739 (+ 224us) server_negotiation.cc:299] Negotiation successful
0811 18:55:31.564859 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260811 18:55:31.569358 10243 tablet_replica.cc:330] T 914a4a144b624c31b6d439067d8173e2 P 485e9ca42b2a45aa9aeeb564daf0edda: stopping tablet replica
I20260811 18:55:31.569749 10243 raft_consensus.cc:2229] T 914a4a144b624c31b6d439067d8173e2 P 485e9ca42b2a45aa9aeeb564daf0edda [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:55:31.570194 10243 raft_consensus.cc:2258] T 914a4a144b624c31b6d439067d8173e2 P 485e9ca42b2a45aa9aeeb564daf0edda [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:55:31.572063 10243 ts_tablet_manager.cc:1905] T 914a4a144b624c31b6d439067d8173e2 P 485e9ca42b2a45aa9aeeb564daf0edda: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:55:31.572790 9421 connection.cc:535] server connection from 127.0.0.1:59056 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 18:55:31.573792 9914 connection.cc:535] server connection from 127.0.0.1:39916 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 18:55:31.574658 10155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:31.542607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54714 (local address 127.27.180.188:34747)
0811 18:55:31.542722 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:31.542728 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:31.565047 (+ 22319us) server_negotiation.cc:407] Connection header received
0811 18:55:31.565571 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:31.565574 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:31.565631 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:31.565681 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:31.566531 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:31.567174 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:31.567946 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:31.568173 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:31.568259 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:31.569609 (+ 1350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:31.569613 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:31.569616 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:31.569642 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:31.571302 (+ 1660us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:31.571817 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:31.571821 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:31.571823 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:31.571909 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:31.572329 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:31.572332 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:31.572334 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:31.572554 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:31.572649 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:31.574358 (+ 1709us) server_negotiation.cc:299] Negotiation successful
0811 18:55:31.574498 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260811 18:55:31.577143 10243 ts_tablet_manager.cc:1918] T 914a4a144b624c31b6d439067d8173e2 P 485e9ca42b2a45aa9aeeb564daf0edda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:55:31.577250 10243 log.cc:1196] T 914a4a144b624c31b6d439067d8173e2 P 485e9ca42b2a45aa9aeeb564daf0edda: Deleting WAL directory at /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/ts-1/wal/wals/914a4a144b624c31b6d439067d8173e2
I20260811 18:55:31.577596 10243 ts_tablet_manager.cc:1939] T 914a4a144b624c31b6d439067d8173e2 P 485e9ca42b2a45aa9aeeb564daf0edda: Deleting consensus metadata
W20260811 18:55:31.579515 9683 connection.cc:441] server connection from 127.0.0.1:58908 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:58908 (request call id 1) could send its response
I20260811 18:55:31.585209 10150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:31.575712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44480 (local address 127.27.180.189:45319)
0811 18:55:31.575814 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:31.575819 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:31.577592 (+ 1773us) server_negotiation.cc:407] Connection header received
0811 18:55:31.577709 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:31.577712 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:31.577760 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:31.577840 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:31.578588 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:31.579358 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:31.580168 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:31.580508 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:31.580618 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:31.581997 (+ 1379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:31.582000 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:31.582003 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:31.582028 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:31.583579 (+ 1551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:31.583943 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:31.583946 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:31.583948 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:31.584051 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:31.584287 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:31.584290 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:31.584292 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:31.584492 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:31.584601 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:31.584926 (+ 325us) server_negotiation.cc:299] Negotiation successful
0811 18:55:31.585030 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260811 18:55:31.588881 10155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:31.581425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54722 (local address 127.27.180.188:34747)
0811 18:55:31.581550 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:31.581554 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:31.581569 (+ 15us) server_negotiation.cc:407] Connection header received
0811 18:55:31.581610 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:31.581613 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:31.581656 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:31.581737 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:31.582479 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:31.583222 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:31.584359 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:31.584666 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:31.584761 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:31.586347 (+ 1586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:31.586351 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:31.586353 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:31.586378 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:31.587570 (+ 1192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:31.587971 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:31.587973 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:31.587975 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:31.588037 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:31.588251 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:31.588253 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:31.588254 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:31.588394 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:31.588477 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:31.588683 (+ 206us) server_negotiation.cc:299] Negotiation successful
0811 18:55:31.588763 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 18:55:32.526002 10256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:32.525334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:46847 (local address 127.27.180.131:50911)
0811 18:55:32.525742 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:32.525851 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:46847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":105,"threads_started":1}
W20260811 18:55:32.526355 10132 heartbeater.cc:644] Failed to heartbeat to 127.27.180.190:46847 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.27.180.190:46847: connect: Connection refused (error 111)
I20260811 18:55:32.528097 10259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:32.527640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:46847 (local address 127.27.180.129:48489)
0811 18:55:32.527903 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:32.528001 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:46847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":72,"threads_started":1}
W20260811 18:55:32.528353 9634 heartbeater.cc:644] Failed to heartbeat to 127.27.180.190:46847 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.27.180.190:46847: connect: Connection refused (error 111)
I20260811 18:55:32.529129 10260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:32.528766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:46847 (local address 127.27.180.130:55949)
0811 18:55:32.528952 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:32.529019 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:46847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":54,"threads_started":1}
W20260811 18:55:32.529348 9867 heartbeater.cc:644] Failed to heartbeat to 127.27.180.190:46847 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.27.180.190:46847: connect: Connection refused (error 111)
I20260811 18:55:33.036515 10261 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9149de133657420baaadd406d6f504fd [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fec531dd027244268997c383dda94895)
I20260811 18:55:33.036727 10261 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9149de133657420baaadd406d6f504fd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "fec531dd027244268997c383dda94895" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 46847 } } peers { permanent_uuid: "e3781302d19a4a8a9342e98ea95ddc93" member_type: VOTER last_known_addr { host: "127.27.180.189" port: 45319 } attrs { promote: false } } peers { permanent_uuid: "9149de133657420baaadd406d6f504fd" member_type: VOTER last_known_addr { host: "127.27.180.188" port: 34747 } attrs { promote: false } }
I20260811 18:55:33.036976 10264 raft_consensus.cc:482] T 00000000000000000000000000000000 P e3781302d19a4a8a9342e98ea95ddc93 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fec531dd027244268997c383dda94895)
I20260811 18:55:33.037179 10264 raft_consensus.cc:504] T 00000000000000000000000000000000 P e3781302d19a4a8a9342e98ea95ddc93 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "fec531dd027244268997c383dda94895" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 46847 } } peers { permanent_uuid: "e3781302d19a4a8a9342e98ea95ddc93" member_type: VOTER last_known_addr { host: "127.27.180.189" port: 45319 } attrs { promote: false } } peers { permanent_uuid: "9149de133657420baaadd406d6f504fd" member_type: VOTER last_known_addr { host: "127.27.180.188" port: 34747 } attrs { promote: false } }
I20260811 18:55:33.037806 10261 leader_election.cc:288] T 00000000000000000000000000000000 P 9149de133657420baaadd406d6f504fd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fec531dd027244268997c383dda94895 (127.27.180.190:46847), e3781302d19a4a8a9342e98ea95ddc93 (127.27.180.189:45319)
I20260811 18:55:33.038096 10264 leader_election.cc:288] T 00000000000000000000000000000000 P e3781302d19a4a8a9342e98ea95ddc93 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fec531dd027244268997c383dda94895 (127.27.180.190:46847), 9149de133657420baaadd406d6f504fd (127.27.180.188:34747)
I20260811 18:55:33.038537 10269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:33.037963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:46847 (local address 127.0.0.1:36686)
0811 18:55:33.038341 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:33.038428 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:46847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":256,"spinlock_wait_cycles":1280,"thread_start_us":88,"threads_started":1}
I20260811 18:55:33.038846 10271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:33.038222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:46847 (local address 127.0.0.1:36702)
0811 18:55:33.038673 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:33.038752 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:46847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":70,"threads_started":1}
W20260811 18:55:33.038889 9269 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.27.180.190:46847: connect: Connection refused (error 111)
W20260811 18:55:33.039022 8915 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.27.180.190:46847: connect: Connection refused (error 111)
I20260811 18:55:33.039891 10269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:33.039656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:46847 (local address 127.0.0.1:36706)
0811 18:55:33.039778 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:33.039827 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:46847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260811 18:55:33.039891 10271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:33.039651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:46847 (local address 127.0.0.1:36708)
0811 18:55:33.039785 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:33.039827 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:46847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
W20260811 18:55:33.040024 9269 leader_election.cc:334] T 00000000000000000000000000000000 P 9149de133657420baaadd406d6f504fd [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer fec531dd027244268997c383dda94895 (127.27.180.190:46847): Network error: Client connection negotiation failed: client connection to 127.27.180.190:46847: connect: Connection refused (error 111)
W20260811 18:55:33.040112 8915 leader_election.cc:334] T 00000000000000000000000000000000 P e3781302d19a4a8a9342e98ea95ddc93 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer fec531dd027244268997c383dda94895 (127.27.180.190:46847): Network error: Client connection negotiation failed: client connection to 127.27.180.190:46847: connect: Connection refused (error 111)
I20260811 18:55:33.043207 10267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:33.038020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.189:45319 (local address 127.0.0.1:44484)
0811 18:55:33.038337 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:33.038369 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:33.038473 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:33.039273 (+ 800us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:33.039275 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:33.039291 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:55:33.039377 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:33.039383 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:33.040442 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:33.040446 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:33.041795 (+ 1349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:33.041807 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:33.042752 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:33.042756 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:33.042871 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:33.042893 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 18:55:33.042980 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 18:55:33.043055 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232,"mutex_wait_us":3,"spinlock_wait_cycles":13568,"thread_start_us":60,"threads_started":1}
I20260811 18:55:33.043190 10272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:33.038342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.188:34747 (local address 127.0.0.1:54728)
0811 18:55:33.038750 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:33.038763 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:33.038860 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:33.039247 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:33.039250 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:33.039265 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:55:33.039369 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:33.039375 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:33.040442 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:33.040446 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:33.041795 (+ 1349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:33.041807 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:33.042752 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:33.042756 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:33.042858 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:33.042885 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 18:55:33.042977 (+ 92us) client_negotiation.cc:240] Negotiation successful
0811 18:55:33.043055 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":73,"threads_started":1}
I20260811 18:55:33.043346 10273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:33.038533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44484 (local address 127.27.180.189:45319)
0811 18:55:33.039060 (+ 527us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:33.039063 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:33.039076 (+ 13us) server_negotiation.cc:407] Connection header received
0811 18:55:33.039116 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:33.039119 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:33.039158 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:33.039470 (+ 312us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:55:33.039552 (+ 82us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:33.040325 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:33.041944 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:33.042600 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:33.042741 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:33.042814 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:33.043044 (+ 230us) server_negotiation.cc:299] Negotiation successful
0811 18:55:33.043130 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":471,"thread_start_us":142,"threads_started":1}
I20260811 18:55:33.043257 10274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:33.038707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54728 (local address 127.27.180.188:34747)
0811 18:55:33.038913 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:33.038917 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:33.038931 (+ 14us) server_negotiation.cc:407] Connection header received
0811 18:55:33.038977 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:33.038980 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:33.039028 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:33.039129 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:55:33.039487 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:33.040325 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:33.041944 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:33.042600 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:33.042743 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:33.042857 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:33.043045 (+ 188us) server_negotiation.cc:299] Negotiation successful
0811 18:55:33.043131 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":44,"server-negotiator.queue_time_us":148,"thread_start_us":64,"threads_started":1}
I20260811 18:55:33.043730 8956 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9149de133657420baaadd406d6f504fd" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "e3781302d19a4a8a9342e98ea95ddc93" is_pre_election: true
I20260811 18:55:33.043745 9316 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e3781302d19a4a8a9342e98ea95ddc93" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "9149de133657420baaadd406d6f504fd" is_pre_election: true
I20260811 18:55:33.043926 8956 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e3781302d19a4a8a9342e98ea95ddc93 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9149de133657420baaadd406d6f504fd in term 1.
I20260811 18:55:33.043923 9316 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9149de133657420baaadd406d6f504fd [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3781302d19a4a8a9342e98ea95ddc93 in term 1.
I20260811 18:55:33.044269 8913 leader_election.cc:302] T 00000000000000000000000000000000 P e3781302d19a4a8a9342e98ea95ddc93 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9149de133657420baaadd406d6f504fd, e3781302d19a4a8a9342e98ea95ddc93; no voters: fec531dd027244268997c383dda94895
I20260811 18:55:33.044293 9260 leader_election.cc:302] T 00000000000000000000000000000000 P 9149de133657420baaadd406d6f504fd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9149de133657420baaadd406d6f504fd, e3781302d19a4a8a9342e98ea95ddc93; no voters: fec531dd027244268997c383dda94895
I20260811 18:55:33.044474 10264 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e3781302d19a4a8a9342e98ea95ddc93 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260811 18:55:33.044529 10264 raft_consensus.cc:482] T 00000000000000000000000000000000 P e3781302d19a4a8a9342e98ea95ddc93 [term 1 FOLLOWER]: Starting leader election (detected failure of leader fec531dd027244268997c383dda94895)
I20260811 18:55:33.044567 10264 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e3781302d19a4a8a9342e98ea95ddc93 [term 1 FOLLOWER]: Advancing to term 2
I20260811 18:55:33.044627 10261 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9149de133657420baaadd406d6f504fd [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260811 18:55:33.044670 10261 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9149de133657420baaadd406d6f504fd [term 1 FOLLOWER]: Starting leader election (detected failure of leader fec531dd027244268997c383dda94895)
I20260811 18:55:33.044704 10261 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9149de133657420baaadd406d6f504fd [term 1 FOLLOWER]: Advancing to term 2
I20260811 18:55:33.045890 10261 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9149de133657420baaadd406d6f504fd [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "fec531dd027244268997c383dda94895" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 46847 } } peers { permanent_uuid: "e3781302d19a4a8a9342e98ea95ddc93" member_type: VOTER last_known_addr { host: "127.27.180.189" port: 45319 } attrs { promote: false } } peers { permanent_uuid: "9149de133657420baaadd406d6f504fd" member_type: VOTER last_known_addr { host: "127.27.180.188" port: 34747 } attrs { promote: false } }
I20260811 18:55:33.045890 10264 raft_consensus.cc:504] T 00000000000000000000000000000000 P e3781302d19a4a8a9342e98ea95ddc93 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "fec531dd027244268997c383dda94895" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 46847 } } peers { permanent_uuid: "e3781302d19a4a8a9342e98ea95ddc93" member_type: VOTER last_known_addr { host: "127.27.180.189" port: 45319 } attrs { promote: false } } peers { permanent_uuid: "9149de133657420baaadd406d6f504fd" member_type: VOTER last_known_addr { host: "127.27.180.188" port: 34747 } attrs { promote: false } }
I20260811 18:55:33.046296 10261 leader_election.cc:288] T 00000000000000000000000000000000 P 9149de133657420baaadd406d6f504fd [CANDIDATE]: Term 2 election: Requested vote from peers fec531dd027244268997c383dda94895 (127.27.180.190:46847), e3781302d19a4a8a9342e98ea95ddc93 (127.27.180.189:45319)
I20260811 18:55:33.046298 10264 leader_election.cc:288] T 00000000000000000000000000000000 P e3781302d19a4a8a9342e98ea95ddc93 [CANDIDATE]: Term 2 election: Requested vote from peers fec531dd027244268997c383dda94895 (127.27.180.190:46847), 9149de133657420baaadd406d6f504fd (127.27.180.188:34747)
I20260811 18:55:33.046753 10267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:33.046488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:46847 (local address 127.0.0.1:36718)
0811 18:55:33.046590 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:33.046648 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:46847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260811 18:55:33.046777 8956 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9149de133657420baaadd406d6f504fd" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "e3781302d19a4a8a9342e98ea95ddc93"
I20260811 18:55:33.046777 9316 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e3781302d19a4a8a9342e98ea95ddc93" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "9149de133657420baaadd406d6f504fd"
I20260811 18:55:33.046931 9316 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 9149de133657420baaadd406d6f504fd [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate e3781302d19a4a8a9342e98ea95ddc93 in current term 2: Already voted for candidate 9149de133657420baaadd406d6f504fd in this term.
I20260811 18:55:33.046938 8956 raft_consensus.cc:2379] T 00000000000000000000000000000000 P e3781302d19a4a8a9342e98ea95ddc93 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 9149de133657420baaadd406d6f504fd in current term 2: Already voted for candidate e3781302d19a4a8a9342e98ea95ddc93 in this term.
I20260811 18:55:33.047429 10267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:33.047195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:46847 (local address 127.0.0.1:36724)
0811 18:55:33.047331 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:33.047377 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:46847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
W20260811 18:55:33.047530 9269 leader_election.cc:334] T 00000000000000000000000000000000 P 9149de133657420baaadd406d6f504fd [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer fec531dd027244268997c383dda94895 (127.27.180.190:46847): Network error: Client connection negotiation failed: client connection to 127.27.180.190:46847: connect: Connection refused (error 111)
I20260811 18:55:33.047600 9269 leader_election.cc:302] T 00000000000000000000000000000000 P 9149de133657420baaadd406d6f504fd [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9149de133657420baaadd406d6f504fd; no voters: e3781302d19a4a8a9342e98ea95ddc93, fec531dd027244268997c383dda94895
I20260811 18:55:33.047694 10272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:33.046742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:46847 (local address 127.0.0.1:36722)
0811 18:55:33.047582 (+ 840us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:33.047633 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:46847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":773}
I20260811 18:55:33.047732 10261 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 9149de133657420baaadd406d6f504fd [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260811 18:55:33.048285 10272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:33.048100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:46847 (local address 127.0.0.1:36730)
0811 18:55:33.048194 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:33.048237 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:46847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 18:55:33.048388 8915 leader_election.cc:334] T 00000000000000000000000000000000 P e3781302d19a4a8a9342e98ea95ddc93 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer fec531dd027244268997c383dda94895 (127.27.180.190:46847): Network error: Client connection negotiation failed: client connection to 127.27.180.190:46847: connect: Connection refused (error 111)
I20260811 18:55:33.048458 8915 leader_election.cc:302] T 00000000000000000000000000000000 P e3781302d19a4a8a9342e98ea95ddc93 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e3781302d19a4a8a9342e98ea95ddc93; no voters: 9149de133657420baaadd406d6f504fd, fec531dd027244268997c383dda94895
I20260811 18:55:33.048579 10264 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e3781302d19a4a8a9342e98ea95ddc93 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260811 18:55:33.527576 10283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:33.527065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:46847 (local address 127.27.180.131:49779)
0811 18:55:33.527404 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:33.527476 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:46847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":104,"threads_started":1}
I20260811 18:55:33.529277 10285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:33.528876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:46847 (local address 127.27.180.129:44569)
0811 18:55:33.529147 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:33.529198 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:46847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":87,"threads_started":1}
I20260811 18:55:33.530118 10286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:33.529782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:46847 (local address 127.27.180.130:36523)
0811 18:55:33.529994 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:33.530044 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:46847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":67,"threads_started":1}
I20260811 18:55:34.529157 10287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:34.528649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:46847 (local address 127.27.180.131:38163)
0811 18:55:34.528939 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:34.529011 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:46847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":79,"threads_started":1}
I20260811 18:55:34.530272 10290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:34.529910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:46847 (local address 127.27.180.129:38417)
0811 18:55:34.530144 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:34.530194 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:46847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":60,"threads_started":1}
I20260811 18:55:34.531039 10291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:34.530683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:46847 (local address 127.27.180.130:53301)
0811 18:55:34.530914 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:34.530959 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:46847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":64,"threads_started":1}
I20260811 18:55:35.099220 10296 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9149de133657420baaadd406d6f504fd [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:55:35.099365 10296 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9149de133657420baaadd406d6f504fd [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "fec531dd027244268997c383dda94895" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 46847 } } peers { permanent_uuid: "e3781302d19a4a8a9342e98ea95ddc93" member_type: VOTER last_known_addr { host: "127.27.180.189" port: 45319 } attrs { promote: false } } peers { permanent_uuid: "9149de133657420baaadd406d6f504fd" member_type: VOTER last_known_addr { host: "127.27.180.188" port: 34747 } attrs { promote: false } }
I20260811 18:55:35.099813 10296 leader_election.cc:288] T 00000000000000000000000000000000 P 9149de133657420baaadd406d6f504fd [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers fec531dd027244268997c383dda94895 (127.27.180.190:46847), e3781302d19a4a8a9342e98ea95ddc93 (127.27.180.189:45319)
I20260811 18:55:35.100322 8956 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9149de133657420baaadd406d6f504fd" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "e3781302d19a4a8a9342e98ea95ddc93" is_pre_election: true
I20260811 18:55:35.100394 10297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:35.100017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:46847 (local address 127.0.0.1:36998)
0811 18:55:35.100242 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:35.100305 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:46847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":58,"threads_started":1}
I20260811 18:55:35.100462 8956 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e3781302d19a4a8a9342e98ea95ddc93 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9149de133657420baaadd406d6f504fd in term 2.
I20260811 18:55:35.100765 9260 leader_election.cc:302] T 00000000000000000000000000000000 P 9149de133657420baaadd406d6f504fd [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9149de133657420baaadd406d6f504fd, e3781302d19a4a8a9342e98ea95ddc93; no voters:
I20260811 18:55:35.100920 10296 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9149de133657420baaadd406d6f504fd [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260811 18:55:35.100981 10296 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9149de133657420baaadd406d6f504fd [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:55:35.101023 10296 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9149de133657420baaadd406d6f504fd [term 2 FOLLOWER]: Advancing to term 3
I20260811 18:55:35.102035 10296 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9149de133657420baaadd406d6f504fd [term 3 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "fec531dd027244268997c383dda94895" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 46847 } } peers { permanent_uuid: "e3781302d19a4a8a9342e98ea95ddc93" member_type: VOTER last_known_addr { host: "127.27.180.189" port: 45319 } attrs { promote: false } } peers { permanent_uuid: "9149de133657420baaadd406d6f504fd" member_type: VOTER last_known_addr { host: "127.27.180.188" port: 34747 } attrs { promote: false } }
I20260811 18:55:35.102415 10296 leader_election.cc:288] T 00000000000000000000000000000000 P 9149de133657420baaadd406d6f504fd [CANDIDATE]: Term 3 election: Requested vote from peers fec531dd027244268997c383dda94895 (127.27.180.190:46847), e3781302d19a4a8a9342e98ea95ddc93 (127.27.180.189:45319)
I20260811 18:55:35.102748 10297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:35.102541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:46847 (local address 127.0.0.1:37006)
0811 18:55:35.102629 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:35.102683 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:46847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260811 18:55:35.102751 8956 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9149de133657420baaadd406d6f504fd" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "e3781302d19a4a8a9342e98ea95ddc93"
I20260811 18:55:35.102861 8956 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e3781302d19a4a8a9342e98ea95ddc93 [term 2 FOLLOWER]: Advancing to term 3
W20260811 18:55:35.102926 9269 leader_election.cc:334] T 00000000000000000000000000000000 P 9149de133657420baaadd406d6f504fd [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer fec531dd027244268997c383dda94895 (127.27.180.190:46847): Network error: Client connection negotiation failed: client connection to 127.27.180.190:46847: connect: Connection refused (error 111)
I20260811 18:55:35.103487 10297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:35.103187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:46847 (local address 127.0.0.1:37018)
0811 18:55:35.103303 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:35.103353 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:46847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260811 18:55:35.103583 9269 leader_election.cc:334] T 00000000000000000000000000000000 P 9149de133657420baaadd406d6f504fd [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer fec531dd027244268997c383dda94895 (127.27.180.190:46847): Network error: Client connection negotiation failed: client connection to 127.27.180.190:46847: connect: Connection refused (error 111)
I20260811 18:55:35.104081 8956 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e3781302d19a4a8a9342e98ea95ddc93 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9149de133657420baaadd406d6f504fd in term 3.
I20260811 18:55:35.104329 9260 leader_election.cc:302] T 00000000000000000000000000000000 P 9149de133657420baaadd406d6f504fd [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9149de133657420baaadd406d6f504fd, e3781302d19a4a8a9342e98ea95ddc93; no voters: fec531dd027244268997c383dda94895
I20260811 18:55:35.104493 10296 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9149de133657420baaadd406d6f504fd [term 3 FOLLOWER]: Leader election won for term 3
I20260811 18:55:35.104693 10296 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9149de133657420baaadd406d6f504fd [term 3 LEADER]: Becoming Leader. State: Replica: 9149de133657420baaadd406d6f504fd, State: Running, Role: LEADER
I20260811 18:55:35.104939 10296 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9149de133657420baaadd406d6f504fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "fec531dd027244268997c383dda94895" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 46847 } } peers { permanent_uuid: "e3781302d19a4a8a9342e98ea95ddc93" member_type: VOTER last_known_addr { host: "127.27.180.189" port: 45319 } attrs { promote: false } } peers { permanent_uuid: "9149de133657420baaadd406d6f504fd" member_type: VOTER last_known_addr { host: "127.27.180.188" port: 34747 } attrs { promote: false } }
I20260811 18:55:35.106518 10302 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9149de133657420baaadd406d6f504fd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9149de133657420baaadd406d6f504fd. Latest consensus state: current_term: 3 leader_uuid: "9149de133657420baaadd406d6f504fd" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "fec531dd027244268997c383dda94895" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 46847 } } peers { permanent_uuid: "e3781302d19a4a8a9342e98ea95ddc93" member_type: VOTER last_known_addr { host: "127.27.180.189" port: 45319 } attrs { promote: false } } peers { permanent_uuid: "9149de133657420baaadd406d6f504fd" member_type: VOTER last_known_addr { host: "127.27.180.188" port: 34747 } attrs { promote: false } } }
I20260811 18:55:35.106629 10302 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9149de133657420baaadd406d6f504fd [sys.catalog]: This master's current role is: LEADER
I20260811 18:55:35.106909 10305 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:55:35.109338 10305 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:55:35.109709 10305 catalog_manager.cc:1259] Loaded cluster ID: c733e261438841f4aa5d657b05032fa9
I20260811 18:55:35.109754 10305 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:55:35.110287 10305 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:55:35.110646 10305 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 9149de133657420baaadd406d6f504fd: Loaded TSK: 0
I20260811 18:55:35.111276 10305 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:55:35.194161 9293 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54722:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:55:35.197188 9293 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:55:35.200276 10297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:35.200040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:46847 (local address 127.0.0.1:37034)
0811 18:55:35.200151 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:35.200208 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:46847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260811 18:55:35.200356 8956 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e3781302d19a4a8a9342e98ea95ddc93 [term 3 FOLLOWER]: Refusing update from remote peer 9149de133657420baaadd406d6f504fd: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260811 18:55:35.200839 10302 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9149de133657420baaadd406d6f504fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3781302d19a4a8a9342e98ea95ddc93" member_type: VOTER last_known_addr { host: "127.27.180.189" port: 45319 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260811 18:55:35.201020 10297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:35.200774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:46847 (local address 127.0.0.1:37046)
0811 18:55:35.200906 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:35.200958 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:46847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260811 18:55:35.201170 9269 consensus_peers.cc:487] T 00000000000000000000000000000000 P 9149de133657420baaadd406d6f504fd -> Peer fec531dd027244268997c383dda94895 (127.27.180.190:46847): Couldn't send request to peer fec531dd027244268997c383dda94895. Status: Network error: Client connection negotiation failed: client connection to 127.27.180.190:46847: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 18:55:35.202280 10308 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3781302d19a4a8a9342e98ea95ddc93 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9149de133657420baaadd406d6f504fd. Latest consensus state: current_term: 3 leader_uuid: "9149de133657420baaadd406d6f504fd" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "fec531dd027244268997c383dda94895" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 46847 } } peers { permanent_uuid: "e3781302d19a4a8a9342e98ea95ddc93" member_type: VOTER last_known_addr { host: "127.27.180.189" port: 45319 } attrs { promote: false } } peers { permanent_uuid: "9149de133657420baaadd406d6f504fd" member_type: VOTER last_known_addr { host: "127.27.180.188" port: 34747 } attrs { promote: false } } }
I20260811 18:55:35.202409 10308 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3781302d19a4a8a9342e98ea95ddc93 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:55:35.203500 10296 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9149de133657420baaadd406d6f504fd [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "9149de133657420baaadd406d6f504fd" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "fec531dd027244268997c383dda94895" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 46847 } } peers { permanent_uuid: "e3781302d19a4a8a9342e98ea95ddc93" member_type: VOTER last_known_addr { host: "127.27.180.189" port: 45319 } attrs { promote: false } } peers { permanent_uuid: "9149de133657420baaadd406d6f504fd" member_type: VOTER last_known_addr { host: "127.27.180.188" port: 34747 } attrs { promote: false } } }
I20260811 18:55:35.203608 10296 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9149de133657420baaadd406d6f504fd [sys.catalog]: This master's current role is: LEADER
I20260811 18:55:35.203889 10308 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3781302d19a4a8a9342e98ea95ddc93 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "9149de133657420baaadd406d6f504fd" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "fec531dd027244268997c383dda94895" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 46847 } } peers { permanent_uuid: "e3781302d19a4a8a9342e98ea95ddc93" member_type: VOTER last_known_addr { host: "127.27.180.189" port: 45319 } attrs { promote: false } } peers { permanent_uuid: "9149de133657420baaadd406d6f504fd" member_type: VOTER last_known_addr { host: "127.27.180.188" port: 34747 } attrs { promote: false } } }
I20260811 18:55:35.204321 10308 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3781302d19a4a8a9342e98ea95ddc93 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:55:35.212994 10297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:35.209022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.131:39783 (local address 127.0.0.1:39930)
0811 18:55:35.209204 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:35.209217 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:35.209301 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:35.209789 (+ 488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:35.209792 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:35.209807 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:55:35.209914 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:35.209921 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:35.210938 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:35.210941 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:35.211973 (+ 1032us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:35.211982 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:35.212635 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:35.212639 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:35.212729 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:35.212758 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 18:55:35.212806 (+ 48us) client_negotiation.cc:240] Negotiation successful
0811 18:55:35.212876 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118}
I20260811 18:55:35.213624 10318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:35.209208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39930 (local address 127.27.180.131:39783)
0811 18:55:35.209540 (+ 332us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:35.209545 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:35.209558 (+ 13us) server_negotiation.cc:407] Connection header received
0811 18:55:35.209615 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:35.209617 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:35.209669 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:35.210024 (+ 355us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:55:35.210149 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:35.210834 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:35.212086 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:35.212497 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:35.213395 (+ 898us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:35.213437 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:35.213468 (+ 31us) server_negotiation.cc:299] Negotiation successful
0811 18:55:35.213524 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":257,"thread_start_us":79,"threads_started":1}
I20260811 18:55:35.213863 10319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:35.209277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.130:45897 (local address 127.0.0.1:58932)
0811 18:55:35.209496 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:35.209509 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:35.209702 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:35.210538 (+ 836us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:35.210541 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:35.210558 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:55:35.210683 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:35.210690 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:35.211439 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:35.211443 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:35.212611 (+ 1168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:35.212622 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:35.213515 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:35.213519 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:35.213605 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:35.213631 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 18:55:35.213676 (+ 45us) client_negotiation.cc:240] Negotiation successful
0811 18:55:35.213740 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"mutex_wait_us":41,"thread_start_us":106,"threads_started":1}
I20260811 18:55:35.214097 10326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:35.209585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.129:32985 (local address 127.0.0.1:59060)
0811 18:55:35.210022 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:35.210035 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:35.210121 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:35.210586 (+ 465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:35.210589 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:35.210602 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:55:35.210711 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:35.210716 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:35.211697 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:35.211701 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:35.212856 (+ 1155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:35.212865 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:35.213717 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:35.213721 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:35.213806 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:35.213830 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 18:55:35.213909 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 18:55:35.213976 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232,"mutex_wait_us":60,"thread_start_us":74,"threads_started":1}
I20260811 18:55:35.214339 9999 tablet_service.cc:1467] Processing CreateTablet for tablet 073668d088654a43a3b527e1e968a782 (DEFAULT_TABLE table=test-table [id=34cd59f682504875b1b1b7178f447aad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:55:35.214479 10322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:35.209540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58932 (local address 127.27.180.130:45897)
0811 18:55:35.210327 (+ 787us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:35.210332 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:35.210346 (+ 14us) server_negotiation.cc:407] Connection header received
0811 18:55:35.210395 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:35.210398 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:35.210443 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:35.210523 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:55:35.210785 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:35.211323 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:35.212749 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:35.213375 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:35.214197 (+ 822us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:35.214244 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:35.214279 (+ 35us) server_negotiation.cc:299] Negotiation successful
0811 18:55:35.214345 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":705,"thread_start_us":82,"threads_started":1}
I20260811 18:55:35.214653 9999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 073668d088654a43a3b527e1e968a782. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:55:35.214928 9755 tablet_service.cc:1467] Processing CreateTablet for tablet 073668d088654a43a3b527e1e968a782 (DEFAULT_TABLE table=test-table [id=34cd59f682504875b1b1b7178f447aad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:55:35.215167 9755 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 073668d088654a43a3b527e1e968a782. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:55:35.216878 10328 tablet_bootstrap.cc:492] T 073668d088654a43a3b527e1e968a782 P 93dbcfa6e0784096bed84ddb3f611598: Bootstrap starting.
I20260811 18:55:35.217114 10327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:35.209772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59060 (local address 127.27.180.129:32985)
0811 18:55:35.209988 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:35.209993 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:35.210273 (+ 280us) server_negotiation.cc:407] Connection header received
0811 18:55:35.210336 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:35.210339 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:35.210403 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:35.210482 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:55:35.210820 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:35.211590 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:35.212990 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:35.213595 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:35.213709 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:35.213757 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:35.216854 (+ 3097us) server_negotiation.cc:299] Negotiation successful
0811 18:55:35.216963 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":69,"threads_started":1}
I20260811 18:55:35.217432 10329 tablet_bootstrap.cc:492] T 073668d088654a43a3b527e1e968a782 P 485e9ca42b2a45aa9aeeb564daf0edda: Bootstrap starting.
I20260811 18:55:35.217617 9500 tablet_service.cc:1467] Processing CreateTablet for tablet 073668d088654a43a3b527e1e968a782 (DEFAULT_TABLE table=test-table [id=34cd59f682504875b1b1b7178f447aad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:55:35.217867 9500 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 073668d088654a43a3b527e1e968a782. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:55:35.218150 10328 tablet_bootstrap.cc:654] T 073668d088654a43a3b527e1e968a782 P 93dbcfa6e0784096bed84ddb3f611598: Neither blocks nor log segments found. Creating new log.
I20260811 18:55:35.218288 10329 tablet_bootstrap.cc:654] T 073668d088654a43a3b527e1e968a782 P 485e9ca42b2a45aa9aeeb564daf0edda: Neither blocks nor log segments found. Creating new log.
I20260811 18:55:35.219306 10328 tablet_bootstrap.cc:492] T 073668d088654a43a3b527e1e968a782 P 93dbcfa6e0784096bed84ddb3f611598: No bootstrap required, opened a new log
I20260811 18:55:35.219360 10329 tablet_bootstrap.cc:492] T 073668d088654a43a3b527e1e968a782 P 485e9ca42b2a45aa9aeeb564daf0edda: No bootstrap required, opened a new log
I20260811 18:55:35.219388 10328 ts_tablet_manager.cc:1397] T 073668d088654a43a3b527e1e968a782 P 93dbcfa6e0784096bed84ddb3f611598: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 18:55:35.219502 10329 ts_tablet_manager.cc:1397] T 073668d088654a43a3b527e1e968a782 P 485e9ca42b2a45aa9aeeb564daf0edda: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:55:35.219764 10328 raft_consensus.cc:348] T 073668d088654a43a3b527e1e968a782 P 93dbcfa6e0784096bed84ddb3f611598 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93dbcfa6e0784096bed84ddb3f611598" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 39783 } } peers { permanent_uuid: "485e9ca42b2a45aa9aeeb564daf0edda" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 45897 } } peers { permanent_uuid: "b1221343a5f8419ab3f58e8e328cc30c" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 32985 } }
I20260811 18:55:35.219877 10328 raft_consensus.cc:374] T 073668d088654a43a3b527e1e968a782 P 93dbcfa6e0784096bed84ddb3f611598 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:55:35.219926 10328 raft_consensus.cc:729] T 073668d088654a43a3b527e1e968a782 P 93dbcfa6e0784096bed84ddb3f611598 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93dbcfa6e0784096bed84ddb3f611598, State: Initialized, Role: FOLLOWER
I20260811 18:55:35.219926 10329 raft_consensus.cc:348] T 073668d088654a43a3b527e1e968a782 P 485e9ca42b2a45aa9aeeb564daf0edda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93dbcfa6e0784096bed84ddb3f611598" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 39783 } } peers { permanent_uuid: "485e9ca42b2a45aa9aeeb564daf0edda" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 45897 } } peers { permanent_uuid: "b1221343a5f8419ab3f58e8e328cc30c" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 32985 } }
I20260811 18:55:35.220027 10329 raft_consensus.cc:374] T 073668d088654a43a3b527e1e968a782 P 485e9ca42b2a45aa9aeeb564daf0edda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:55:35.220057 10329 raft_consensus.cc:729] T 073668d088654a43a3b527e1e968a782 P 485e9ca42b2a45aa9aeeb564daf0edda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 485e9ca42b2a45aa9aeeb564daf0edda, State: Initialized, Role: FOLLOWER
I20260811 18:55:35.220049 10328 consensus_queue.cc:260] T 073668d088654a43a3b527e1e968a782 P 93dbcfa6e0784096bed84ddb3f611598 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93dbcfa6e0784096bed84ddb3f611598" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 39783 } } peers { permanent_uuid: "485e9ca42b2a45aa9aeeb564daf0edda" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 45897 } } peers { permanent_uuid: "b1221343a5f8419ab3f58e8e328cc30c" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 32985 } }
I20260811 18:55:35.220193 10329 consensus_queue.cc:260] T 073668d088654a43a3b527e1e968a782 P 485e9ca42b2a45aa9aeeb564daf0edda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93dbcfa6e0784096bed84ddb3f611598" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 39783 } } peers { permanent_uuid: "485e9ca42b2a45aa9aeeb564daf0edda" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 45897 } } peers { permanent_uuid: "b1221343a5f8419ab3f58e8e328cc30c" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 32985 } }
I20260811 18:55:35.220379 10328 ts_tablet_manager.cc:1428] T 073668d088654a43a3b527e1e968a782 P 93dbcfa6e0784096bed84ddb3f611598: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 18:55:35.220492 10329 ts_tablet_manager.cc:1428] T 073668d088654a43a3b527e1e968a782 P 485e9ca42b2a45aa9aeeb564daf0edda: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 18:55:35.220590 10332 tablet_bootstrap.cc:492] T 073668d088654a43a3b527e1e968a782 P b1221343a5f8419ab3f58e8e328cc30c: Bootstrap starting.
I20260811 18:55:35.221711 10332 tablet_bootstrap.cc:654] T 073668d088654a43a3b527e1e968a782 P b1221343a5f8419ab3f58e8e328cc30c: Neither blocks nor log segments found. Creating new log.
I20260811 18:55:35.221974 10344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:35.220977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:46847 (local address 127.27.180.131:36403)
0811 18:55:35.221807 (+ 830us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:35.221865 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:46847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":760,"thread_start_us":109,"threads_started":1}
I20260811 18:55:35.222311 10345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:35.221350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:46847 (local address 127.27.180.130:36851)
0811 18:55:35.222166 (+ 816us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:35.222221 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:46847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":748,"thread_start_us":82,"threads_started":1}
I20260811 18:55:35.223050 10332 tablet_bootstrap.cc:492] T 073668d088654a43a3b527e1e968a782 P b1221343a5f8419ab3f58e8e328cc30c: No bootstrap required, opened a new log
I20260811 18:55:35.223127 10131 heartbeater.cc:503] Master 127.27.180.188:34747 was elected leader, sending a full tablet report...
I20260811 18:55:35.223179 10332 ts_tablet_manager.cc:1397] T 073668d088654a43a3b527e1e968a782 P b1221343a5f8419ab3f58e8e328cc30c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 18:55:35.223315 10344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:35.223043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:46847 (local address 127.27.180.131:53631)
0811 18:55:35.223200 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:35.223251 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:46847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260811 18:55:35.223624 10332 raft_consensus.cc:348] T 073668d088654a43a3b527e1e968a782 P b1221343a5f8419ab3f58e8e328cc30c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93dbcfa6e0784096bed84ddb3f611598" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 39783 } } peers { permanent_uuid: "485e9ca42b2a45aa9aeeb564daf0edda" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 45897 } } peers { permanent_uuid: "b1221343a5f8419ab3f58e8e328cc30c" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 32985 } }
I20260811 18:55:35.223789 10332 raft_consensus.cc:374] T 073668d088654a43a3b527e1e968a782 P b1221343a5f8419ab3f58e8e328cc30c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:55:35.223871 10332 raft_consensus.cc:729] T 073668d088654a43a3b527e1e968a782 P b1221343a5f8419ab3f58e8e328cc30c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1221343a5f8419ab3f58e8e328cc30c, State: Initialized, Role: FOLLOWER
I20260811 18:55:35.224120 10345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:35.223858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:46847 (local address 127.27.180.130:40493)
0811 18:55:35.224016 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:35.224062 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:46847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260811 18:55:35.224066 10332 consensus_queue.cc:260] T 073668d088654a43a3b527e1e968a782 P b1221343a5f8419ab3f58e8e328cc30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93dbcfa6e0784096bed84ddb3f611598" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 39783 } } peers { permanent_uuid: "485e9ca42b2a45aa9aeeb564daf0edda" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 45897 } } peers { permanent_uuid: "b1221343a5f8419ab3f58e8e328cc30c" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 32985 } }
I20260811 18:55:35.226084 10332 ts_tablet_manager.cc:1428] T 073668d088654a43a3b527e1e968a782 P b1221343a5f8419ab3f58e8e328cc30c: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260811 18:55:35.226222 9633 heartbeater.cc:503] Master 127.27.180.188:34747 was elected leader, sending a full tablet report...
I20260811 18:55:35.226432 9866 heartbeater.cc:503] Master 127.27.180.188:34747 was elected leader, sending a full tablet report...
I20260811 18:55:35.226804 10351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:35.224893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:46847 (local address 127.27.180.129:40471)
0811 18:55:35.226662 (+ 1769us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:35.226717 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:46847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1713,"thread_start_us":93,"threads_started":1}
I20260811 18:55:35.227461 10351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:35.227216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:46847 (local address 127.27.180.129:44165)
0811 18:55:35.227319 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:35.227371 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:46847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260811 18:55:35.310626 10136 tablet.cc:2369] T 073668d088654a43a3b527e1e968a782 P 93dbcfa6e0784096bed84ddb3f611598: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 18:55:35.338466 9871 tablet.cc:2369] T 073668d088654a43a3b527e1e968a782 P 485e9ca42b2a45aa9aeeb564daf0edda: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 18:55:35.352890 9637 tablet.cc:2369] T 073668d088654a43a3b527e1e968a782 P b1221343a5f8419ab3f58e8e328cc30c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:55:35.550431 10350 raft_consensus.cc:482] T 073668d088654a43a3b527e1e968a782 P b1221343a5f8419ab3f58e8e328cc30c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:55:35.550587 10350 raft_consensus.cc:504] T 073668d088654a43a3b527e1e968a782 P b1221343a5f8419ab3f58e8e328cc30c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93dbcfa6e0784096bed84ddb3f611598" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 39783 } } peers { permanent_uuid: "485e9ca42b2a45aa9aeeb564daf0edda" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 45897 } } peers { permanent_uuid: "b1221343a5f8419ab3f58e8e328cc30c" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 32985 } }
I20260811 18:55:35.552033 10350 leader_election.cc:288] T 073668d088654a43a3b527e1e968a782 P b1221343a5f8419ab3f58e8e328cc30c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 93dbcfa6e0784096bed84ddb3f611598 (127.27.180.131:39783), 485e9ca42b2a45aa9aeeb564daf0edda (127.27.180.130:45897)
I20260811 18:55:35.555740 10334 raft_consensus.cc:482] T 073668d088654a43a3b527e1e968a782 P 93dbcfa6e0784096bed84ddb3f611598 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:55:35.555871 10334 raft_consensus.cc:504] T 073668d088654a43a3b527e1e968a782 P 93dbcfa6e0784096bed84ddb3f611598 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93dbcfa6e0784096bed84ddb3f611598" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 39783 } } peers { permanent_uuid: "485e9ca42b2a45aa9aeeb564daf0edda" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 45897 } } peers { permanent_uuid: "b1221343a5f8419ab3f58e8e328cc30c" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 32985 } }
I20260811 18:55:35.556129 10318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:35.552630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.129:57501 (local address 127.27.180.131:39783)
0811 18:55:35.552767 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:35.552771 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:35.552782 (+ 11us) server_negotiation.cc:407] Connection header received
0811 18:55:35.552827 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:35.552829 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:35.552867 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:35.552939 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:55:35.553250 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:35.553922 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:35.555087 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:35.555486 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:35.555588 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:35.555647 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:35.555925 (+ 278us) server_negotiation.cc:299] Negotiation successful
0811 18:55:35.556005 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 18:55:35.556154 10356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:35.552279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.130:45897 (local address 127.27.180.129:47361)
0811 18:55:35.552678 (+ 399us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:35.552691 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:35.552801 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:35.553019 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:35.553021 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:35.553037 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:55:35.553183 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:35.553189 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:35.554138 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:35.554142 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:35.554962 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:35.554971 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:35.555626 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:35.555630 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:35.555793 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:35.555816 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 18:55:35.555887 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 18:55:35.555967 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"mutex_wait_us":69,"thread_start_us":121,"threads_started":1}
I20260811 18:55:35.556473 10351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:35.552264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.131:39783 (local address 127.27.180.129:57501)
0811 18:55:35.552518 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:35.552533 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:35.552638 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:35.552962 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:35.552965 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:35.552982 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:55:35.553125 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:35.553132 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:35.554021 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:35.554024 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:35.554974 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:35.554983 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:35.555609 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:35.555613 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:35.555757 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:35.555799 (+ 42us) client_negotiation.cc:769] Sending connection context
0811 18:55:35.556271 (+ 472us) client_negotiation.cc:240] Negotiation successful
0811 18:55:35.556351 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"mutex_wait_us":92}
I20260811 18:55:35.556486 10322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:35.552390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.129:47361 (local address 127.27.180.130:45897)
0811 18:55:35.552511 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:35.552516 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:35.552760 (+ 244us) server_negotiation.cc:407] Connection header received
0811 18:55:35.552888 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:35.552891 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:35.552941 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:35.553009 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:55:35.553290 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:35.554042 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:35.555077 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:35.555484 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:35.555595 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:35.555647 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:35.556312 (+ 665us) server_negotiation.cc:299] Negotiation successful
0811 18:55:35.556390 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260811 18:55:35.556964 9787 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "073668d088654a43a3b527e1e968a782" candidate_uuid: "b1221343a5f8419ab3f58e8e328cc30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "485e9ca42b2a45aa9aeeb564daf0edda" is_pre_election: true
I20260811 18:55:35.557175 10032 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "073668d088654a43a3b527e1e968a782" candidate_uuid: "b1221343a5f8419ab3f58e8e328cc30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93dbcfa6e0784096bed84ddb3f611598" is_pre_election: true
I20260811 18:55:35.557205 9787 raft_consensus.cc:2454] T 073668d088654a43a3b527e1e968a782 P 485e9ca42b2a45aa9aeeb564daf0edda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1221343a5f8419ab3f58e8e328cc30c in term 0.
I20260811 18:55:35.557302 10032 raft_consensus.cc:2454] T 073668d088654a43a3b527e1e968a782 P 93dbcfa6e0784096bed84ddb3f611598 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1221343a5f8419ab3f58e8e328cc30c in term 0.
I20260811 18:55:35.557585 9423 leader_election.cc:302] T 073668d088654a43a3b527e1e968a782 P b1221343a5f8419ab3f58e8e328cc30c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 485e9ca42b2a45aa9aeeb564daf0edda, b1221343a5f8419ab3f58e8e328cc30c; no voters:
I20260811 18:55:35.557753 10334 leader_election.cc:288] T 073668d088654a43a3b527e1e968a782 P 93dbcfa6e0784096bed84ddb3f611598 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 485e9ca42b2a45aa9aeeb564daf0edda (127.27.180.130:45897), b1221343a5f8419ab3f58e8e328cc30c (127.27.180.129:32985)
I20260811 18:55:35.557864 10350 raft_consensus.cc:2781] T 073668d088654a43a3b527e1e968a782 P b1221343a5f8419ab3f58e8e328cc30c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:55:35.557932 10350 raft_consensus.cc:482] T 073668d088654a43a3b527e1e968a782 P b1221343a5f8419ab3f58e8e328cc30c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:55:35.558009 10350 raft_consensus.cc:3037] T 073668d088654a43a3b527e1e968a782 P b1221343a5f8419ab3f58e8e328cc30c [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:55:35.559387 10350 raft_consensus.cc:504] T 073668d088654a43a3b527e1e968a782 P b1221343a5f8419ab3f58e8e328cc30c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93dbcfa6e0784096bed84ddb3f611598" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 39783 } } peers { permanent_uuid: "485e9ca42b2a45aa9aeeb564daf0edda" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 45897 } } peers { permanent_uuid: "b1221343a5f8419ab3f58e8e328cc30c" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 32985 } }
I20260811 18:55:35.560046 10032 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "073668d088654a43a3b527e1e968a782" candidate_uuid: "b1221343a5f8419ab3f58e8e328cc30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93dbcfa6e0784096bed84ddb3f611598"
I20260811 18:55:35.560158 10032 raft_consensus.cc:3037] T 073668d088654a43a3b527e1e968a782 P 93dbcfa6e0784096bed84ddb3f611598 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:55:35.560158 9787 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "073668d088654a43a3b527e1e968a782" candidate_uuid: "b1221343a5f8419ab3f58e8e328cc30c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "485e9ca42b2a45aa9aeeb564daf0edda"
I20260811 18:55:35.560245 9787 raft_consensus.cc:3037] T 073668d088654a43a3b527e1e968a782 P 485e9ca42b2a45aa9aeeb564daf0edda [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:55:35.561249 10032 raft_consensus.cc:2454] T 073668d088654a43a3b527e1e968a782 P 93dbcfa6e0784096bed84ddb3f611598 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1221343a5f8419ab3f58e8e328cc30c in term 1.
I20260811 18:55:35.561264 9787 raft_consensus.cc:2454] T 073668d088654a43a3b527e1e968a782 P 485e9ca42b2a45aa9aeeb564daf0edda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1221343a5f8419ab3f58e8e328cc30c in term 1.
I20260811 18:55:35.561478 10350 leader_election.cc:288] T 073668d088654a43a3b527e1e968a782 P b1221343a5f8419ab3f58e8e328cc30c [CANDIDATE]: Term 1 election: Requested vote from peers 93dbcfa6e0784096bed84ddb3f611598 (127.27.180.131:39783), 485e9ca42b2a45aa9aeeb564daf0edda (127.27.180.130:45897)
I20260811 18:55:35.561816 9423 leader_election.cc:302] T 073668d088654a43a3b527e1e968a782 P b1221343a5f8419ab3f58e8e328cc30c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 485e9ca42b2a45aa9aeeb564daf0edda, b1221343a5f8419ab3f58e8e328cc30c; no voters:
I20260811 18:55:35.562077 10322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:35.558502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.131:38765 (local address 127.27.180.130:45897)
0811 18:55:35.558609 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:35.558613 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:35.558624 (+ 11us) server_negotiation.cc:407] Connection header received
0811 18:55:35.558658 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:35.558660 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:35.558690 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:35.558748 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:55:35.558973 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:35.559410 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:35.560797 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:35.561395 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:35.561490 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:35.561535 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:35.561920 (+ 385us) server_negotiation.cc:299] Negotiation successful
0811 18:55:35.561984 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 18:55:35.562078 10344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:35.557898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.130:45897 (local address 127.27.180.131:38765)
0811 18:55:35.558033 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:35.558046 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:35.558131 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:35.558769 (+ 638us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:35.558772 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:35.558790 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:55:35.558880 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:35.558884 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:35.559509 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:35.559512 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:35.560670 (+ 1158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:35.560679 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:35.561621 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:35.561625 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:35.561765 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:35.561787 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 18:55:35.561890 (+ 103us) client_negotiation.cc:240] Negotiation successful
0811 18:55:35.561957 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65,"mutex_wait_us":1}
I20260811 18:55:35.562162 10350 raft_consensus.cc:2781] T 073668d088654a43a3b527e1e968a782 P b1221343a5f8419ab3f58e8e328cc30c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:55:35.562345 10350 raft_consensus.cc:686] T 073668d088654a43a3b527e1e968a782 P b1221343a5f8419ab3f58e8e328cc30c [term 1 LEADER]: Becoming Leader. State: Replica: b1221343a5f8419ab3f58e8e328cc30c, State: Running, Role: LEADER
I20260811 18:55:35.562434 9787 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "073668d088654a43a3b527e1e968a782" candidate_uuid: "93dbcfa6e0784096bed84ddb3f611598" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "485e9ca42b2a45aa9aeeb564daf0edda" is_pre_election: true
I20260811 18:55:35.562601 9787 raft_consensus.cc:2379] T 073668d088654a43a3b527e1e968a782 P 485e9ca42b2a45aa9aeeb564daf0edda [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 93dbcfa6e0784096bed84ddb3f611598 in current term 1: Already voted for candidate b1221343a5f8419ab3f58e8e328cc30c in this term.
I20260811 18:55:35.562615 10350 consensus_queue.cc:237] T 073668d088654a43a3b527e1e968a782 P b1221343a5f8419ab3f58e8e328cc30c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93dbcfa6e0784096bed84ddb3f611598" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 39783 } } peers { permanent_uuid: "485e9ca42b2a45aa9aeeb564daf0edda" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 45897 } } peers { permanent_uuid: "b1221343a5f8419ab3f58e8e328cc30c" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 32985 } }
I20260811 18:55:35.562906 10327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:35.558533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.131:34451 (local address 127.27.180.129:32985)
0811 18:55:35.558622 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:35.558625 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:35.558636 (+ 11us) server_negotiation.cc:407] Connection header received
0811 18:55:35.558671 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:35.558674 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:35.558709 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:35.558774 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:55:35.559759 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:35.560469 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:35.561888 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:35.562411 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:35.562519 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:35.562560 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:35.562750 (+ 190us) server_negotiation.cc:299] Negotiation successful
0811 18:55:35.562815 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260811 18:55:35.562945 10358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:35.558146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.129:32985 (local address 127.27.180.131:34451)
0811 18:55:35.558438 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:35.558450 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:35.558533 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:35.559491 (+ 958us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:35.559494 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:35.559516 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:55:35.559628 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:35.559634 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:35.560593 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:35.560597 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:35.561778 (+ 1181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:35.561786 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:35.562517 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:35.562520 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:35.562624 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:35.562649 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 18:55:35.562726 (+ 77us) client_negotiation.cc:240] Negotiation successful
0811 18:55:35.562790 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":226,"mutex_wait_us":25,"thread_start_us":109,"threads_started":1}
I20260811 18:55:35.563580 9535 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "073668d088654a43a3b527e1e968a782" candidate_uuid: "93dbcfa6e0784096bed84ddb3f611598" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1221343a5f8419ab3f58e8e328cc30c" is_pre_election: true
I20260811 18:55:35.563971 9911 leader_election.cc:302] T 073668d088654a43a3b527e1e968a782 P 93dbcfa6e0784096bed84ddb3f611598 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 93dbcfa6e0784096bed84ddb3f611598; no voters: 485e9ca42b2a45aa9aeeb564daf0edda, b1221343a5f8419ab3f58e8e328cc30c
I20260811 18:55:35.564174 10334 raft_consensus.cc:2726] T 073668d088654a43a3b527e1e968a782 P 93dbcfa6e0784096bed84ddb3f611598 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 18:55:35.564781 10351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:35.564553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:46847 (local address 127.27.180.129:46125)
0811 18:55:35.564674 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:35.564723 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:46847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260811 18:55:35.565078 9278 catalog_manager.cc:5495] T 073668d088654a43a3b527e1e968a782 P b1221343a5f8419ab3f58e8e328cc30c reported cstate change: term changed from 0 to 1, leader changed from <none> to b1221343a5f8419ab3f58e8e328cc30c (127.27.180.129). New cstate: current_term: 1 leader_uuid: "b1221343a5f8419ab3f58e8e328cc30c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93dbcfa6e0784096bed84ddb3f611598" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 39783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "485e9ca42b2a45aa9aeeb564daf0edda" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 45897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1221343a5f8419ab3f58e8e328cc30c" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 32985 } health_report { overall_health: HEALTHY } } }
I20260811 18:55:35.646418 10326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:35.646126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:46847 (local address 127.0.0.1:37050)
0811 18:55:35.646266 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:35.646330 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:46847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260811 18:55:35.647110 10326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:35.646898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:46847 (local address 127.0.0.1:37058)
0811 18:55:35.647002 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:35.647049 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:46847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260811 18:55:35.661531 10327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:35.658536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59068 (local address 127.27.180.129:32985)
0811 18:55:35.658649 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:35.658653 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:35.658665 (+ 12us) server_negotiation.cc:407] Connection header received
0811 18:55:35.658738 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:35.658741 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:35.658797 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:35.658890 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:55:35.659062 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:35.659563 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:35.660317 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:35.660537 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:35.660614 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:35.660788 (+ 174us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:55:35.661245 (+ 457us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:55:35.661331 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:35.661388 (+ 57us) server_negotiation.cc:299] Negotiation successful
0811 18:55:35.661434 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 18:55:35.664122 10032 raft_consensus.cc:1261] T 073668d088654a43a3b527e1e968a782 P 93dbcfa6e0784096bed84ddb3f611598 [term 1 FOLLOWER]: Refusing update from remote peer b1221343a5f8419ab3f58e8e328cc30c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:55:35.664309 9787 raft_consensus.cc:1261] T 073668d088654a43a3b527e1e968a782 P 485e9ca42b2a45aa9aeeb564daf0edda [term 1 FOLLOWER]: Refusing update from remote peer b1221343a5f8419ab3f58e8e328cc30c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:55:35.664623 10359 consensus_queue.cc:1035] T 073668d088654a43a3b527e1e968a782 P b1221343a5f8419ab3f58e8e328cc30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "93dbcfa6e0784096bed84ddb3f611598" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 39783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:55:35.664798 10350 consensus_queue.cc:1035] T 073668d088654a43a3b527e1e968a782 P b1221343a5f8419ab3f58e8e328cc30c [LEADER]: Connected to new peer: Peer: permanent_uuid: "485e9ca42b2a45aa9aeeb564daf0edda" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 45897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:55:35.666657 10345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:35.666408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:46847 (local address 127.27.180.130:56689)
0811 18:55:35.666526 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:35.666585 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:46847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260811 18:55:35.668229 10344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:35.667963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:46847 (local address 127.27.180.131:59863)
0811 18:55:35.668109 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:35.668161 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:46847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260811 18:55:35.669883 10351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:35.669484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:46847 (local address 127.27.180.129:37991)
0811 18:55:35.669766 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:35.669818 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:46847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215}
I20260811 18:55:35.669907 10345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:35.669654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:46847 (local address 127.27.180.130:55151)
0811 18:55:35.669789 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:35.669840 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:46847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260811 18:55:35.670970 10367 mvcc.cc:205] Tried to move back new op lower bound from 7317399698075750400 to 7317399697667149824. Current Snapshot: MvccSnapshot[applied={T|T < 7317399698075750400}]
I20260811 18:55:35.672149 10351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:35.671934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:46847 (local address 127.27.180.129:40057)
0811 18:55:35.672022 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:35.672067 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:46847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260811 18:55:35.677127 10344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:35.676744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:46847 (local address 127.27.180.131:55079)
0811 18:55:35.676977 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:35.677030 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:46847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167}
I20260811 18:55:35.677750 9278 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54722:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:55:35.678009 9278 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54722:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:55:35.681586 9278 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9149de133657420baaadd406d6f504fd: Sending DeleteTablet for 3 replicas of tablet 073668d088654a43a3b527e1e968a782
I20260811 18:55:35.682145 9999 tablet_service.cc:1514] Processing DeleteTablet for tablet 073668d088654a43a3b527e1e968a782 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:55:35 UTC) from {username='oryx', principal='oryx/127.27.180.188@KRBTEST.COM'} at 127.0.0.1:39930
I20260811 18:55:35.682433 9500 tablet_service.cc:1514] Processing DeleteTablet for tablet 073668d088654a43a3b527e1e968a782 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:55:35 UTC) from {username='oryx', principal='oryx/127.27.180.188@KRBTEST.COM'} at 127.0.0.1:59060
I20260811 18:55:35.682739 10380 tablet_replica.cc:330] T 073668d088654a43a3b527e1e968a782 P b1221343a5f8419ab3f58e8e328cc30c: stopping tablet replica
I20260811 18:55:35.682754 9755 tablet_service.cc:1514] Processing DeleteTablet for tablet 073668d088654a43a3b527e1e968a782 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:55:35 UTC) from {username='oryx', principal='oryx/127.27.180.188@KRBTEST.COM'} at 127.0.0.1:58932
I20260811 18:55:35.682914 10380 raft_consensus.cc:2229] T 073668d088654a43a3b527e1e968a782 P b1221343a5f8419ab3f58e8e328cc30c [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:55:35.683182 10379 tablet_replica.cc:330] T 073668d088654a43a3b527e1e968a782 P 93dbcfa6e0784096bed84ddb3f611598: stopping tablet replica
I20260811 18:55:35.683187 10380 raft_consensus.cc:2258] T 073668d088654a43a3b527e1e968a782 P b1221343a5f8419ab3f58e8e328cc30c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:55:35.683359 10379 raft_consensus.cc:2229] T 073668d088654a43a3b527e1e968a782 P 93dbcfa6e0784096bed84ddb3f611598 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:55:35.683524 10379 raft_consensus.cc:2258] T 073668d088654a43a3b527e1e968a782 P 93dbcfa6e0784096bed84ddb3f611598 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:55:35.683827 10380 ts_tablet_manager.cc:1905] T 073668d088654a43a3b527e1e968a782 P b1221343a5f8419ab3f58e8e328cc30c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:55:35.683858 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 9215
I20260811 18:55:35.689463 10379 ts_tablet_manager.cc:1905] T 073668d088654a43a3b527e1e968a782 P 93dbcfa6e0784096bed84ddb3f611598: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:55:35.691892 10379 ts_tablet_manager.cc:1918] T 073668d088654a43a3b527e1e968a782 P 93dbcfa6e0784096bed84ddb3f611598: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:55:35.691998 10379 log.cc:1196] T 073668d088654a43a3b527e1e968a782 P 93dbcfa6e0784096bed84ddb3f611598: Deleting WAL directory at /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/ts-2/wal/wals/073668d088654a43a3b527e1e968a782
I20260811 18:55:35.692219 10381 tablet_replica.cc:330] T 073668d088654a43a3b527e1e968a782 P 485e9ca42b2a45aa9aeeb564daf0edda: stopping tablet replica
I20260811 18:55:35.692355 10379 ts_tablet_manager.cc:1939] T 073668d088654a43a3b527e1e968a782 P 93dbcfa6e0784096bed84ddb3f611598: Deleting consensus metadata
I20260811 18:55:35.692373 10381 raft_consensus.cc:2229] T 073668d088654a43a3b527e1e968a782 P 485e9ca42b2a45aa9aeeb564daf0edda [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:55:35.692493 10381 raft_consensus.cc:2258] T 073668d088654a43a3b527e1e968a782 P 485e9ca42b2a45aa9aeeb564daf0edda [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:55:35.693953 9260 catalog_manager.cc:4841] TS 93dbcfa6e0784096bed84ddb3f611598 (127.27.180.131:39783): tablet 073668d088654a43a3b527e1e968a782 (table test-table [id=34cd59f682504875b1b1b7178f447aad]) successfully deleted
I20260811 18:55:35.694168 10381 ts_tablet_manager.cc:1905] T 073668d088654a43a3b527e1e968a782 P 485e9ca42b2a45aa9aeeb564daf0edda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:55:35.696132 10381 ts_tablet_manager.cc:1918] T 073668d088654a43a3b527e1e968a782 P 485e9ca42b2a45aa9aeeb564daf0edda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:55:35.696201 10381 log.cc:1196] T 073668d088654a43a3b527e1e968a782 P 485e9ca42b2a45aa9aeeb564daf0edda: Deleting WAL directory at /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786474496157106-28370-0/minicluster-data/ts-1/wal/wals/073668d088654a43a3b527e1e968a782
I20260811 18:55:35.696444 10381 ts_tablet_manager.cc:1939] T 073668d088654a43a3b527e1e968a782 P 485e9ca42b2a45aa9aeeb564daf0edda: Deleting consensus metadata
I20260811 18:55:35.697786 9261 catalog_manager.cc:4841] TS 485e9ca42b2a45aa9aeeb564daf0edda (127.27.180.130:45897): tablet 073668d088654a43a3b527e1e968a782 (table test-table [id=34cd59f682504875b1b1b7178f447aad]) successfully deleted
I20260811 18:55:35.703754 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 9648
I20260811 18:55:35.703922 10326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:35.703623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.129:32985 (local address 127.0.0.1:59070)
0811 18:55:35.703764 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:35.703837 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.129:32985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260811 18:55:35.705893 9261 catalog_manager.cc:4584] TS b1221343a5f8419ab3f58e8e328cc30c (127.27.180.129:32985): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 073668d088654a43a3b527e1e968a782: Network error: Client connection negotiation failed: client connection to 127.27.180.129:32985: connect: Connection refused (error 111)
I20260811 18:55:35.715709 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 9877
I20260811 18:55:35.730619 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 8659
I20260811 18:55:35.740398 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 8875
2026-08-11T18:55:35Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (7970 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:55:35 dist-test-slave-2j7r krb5kdc[10385](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:55:35 dist-test-slave-2j7r krb5kdc[10385](info): set up 2 sockets
Aug 11 18:55:35 dist-test-slave-2j7r krb5kdc[10385](info): commencing operation
krb5kdc: starting...
W20260811 18:55:36.968884 28370 mini_kdc.cc:121] Time spent starting KDC: real 1.195s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/krb5kdc/test-user.keytab.
Aug 11 18:55:37 dist-test-slave-2j7r krb5kdc[10385](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474537, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:55:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:55:37Z Disabled control of system clock
WARNING: no policy specified for oryx/127.27.180.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.27.180.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.27.180.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/krb5kdc/oryx_127.27.180.190.keytab.
Entry for principal oryx/127.27.180.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/krb5kdc/oryx_127.27.180.190.keytab.
I20260811 18:55:37.092092 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.190:37195
--webserver_interface=127.27.180.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/krb5kdc/oryx_127.27.180.190.keytab
--principal=oryx/127.27.180.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:42439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.180.190:37195
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:55:37.203642 10401 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:55:37.203920 10401 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:55:37.203976 10401 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:55:37.208031 10401 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:55:37.208093 10401 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 18:55:37.208117 10401 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:55:37.208139 10401 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:55:37.208161 10401 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:55:37.208190 10401 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:55:37.213581 10401 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:42439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.180.190:37195
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/krb5kdc/oryx_127.27.180.190.keytab
--principal=oryx/127.27.180.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.190:37195
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.180.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10401
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:55:37.214666 10401 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:55:37.215705 10401 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:55:37.224500 10411 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:37.224489 10415 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:55:37.224535 10401 server_base.cc:1034] running on GCE node
W20260811 18:55:37.224489 10409 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:55:37.225299 10401 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:55:37.226398 10401 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:55:37.227560 10401 hybrid_clock.cc:648] HybridClock initialized: now 1786474537227550 us; error 27 us; skew 500 ppm
Aug 11 18:55:37 dist-test-slave-2j7r krb5kdc[10385](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474537, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.180.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:55:37.230286 10401 init.cc:377] Logged in from keytab as oryx/127.27.180.190@KRBTEST.COM (short username oryx)
I20260811 18:55:37.231168 10401 webserver.cc:460] Webserver started at http://127.27.180.190:41273/ using document root <none> and password file <none>
I20260811 18:55:37.231722 10401 fs_manager.cc:359] Metadata directory not provided
I20260811 18:55:37.231786 10401 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:55:37.231992 10401 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:55:37.234102 10401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/minicluster-data/master-0/data/instance:
uuid: "8627c387211147ed82345656961a4525"
format_stamp: "Formatted at 2026-08-11 18:55:37 on dist-test-slave-2j7r"
server_key: "bdd58094125ba305017ffaea09ae8eb1"
server_key_iv: "9143a26aa128b255a1e37387b50258c7"
server_key_version: "encryptionkey@0"
I20260811 18:55:37.234658 10401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/minicluster-data/master-0/wal/instance:
uuid: "8627c387211147ed82345656961a4525"
format_stamp: "Formatted at 2026-08-11 18:55:37 on dist-test-slave-2j7r"
server_key: "bdd58094125ba305017ffaea09ae8eb1"
server_key_iv: "9143a26aa128b255a1e37387b50258c7"
server_key_version: "encryptionkey@0"
I20260811 18:55:37.238476 10401 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 18:55:37.240849 10426 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:37.242200 10401 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 18:55:37.242307 10401 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/minicluster-data/master-0/wal
uuid: "8627c387211147ed82345656961a4525"
format_stamp: "Formatted at 2026-08-11 18:55:37 on dist-test-slave-2j7r"
server_key: "bdd58094125ba305017ffaea09ae8eb1"
server_key_iv: "9143a26aa128b255a1e37387b50258c7"
server_key_version: "encryptionkey@0"
I20260811 18:55:37.242393 10401 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:55:37.263530 10401 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:55:37.266340 10401 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:55:37.266521 10401 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:55:37.274119 10401 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.190:37195
I20260811 18:55:37.274134 10540 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.190:37195 every 8 connection(s)
I20260811 18:55:37.275044 10401 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
I20260811 18:55:37.277737 10544 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:55:37.278357 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 10401
I20260811 18:55:37.278478 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/minicluster-data/master-0/wal/instance
I20260811 18:55:37.278797 28370 external_mini_cluster.cc:1428] Setting key 97ffaabe3871892f2b55d0c02384a49b
I20260811 18:55:37.283829 10544 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8627c387211147ed82345656961a4525: Bootstrap starting.
I20260811 18:55:37.286126 10544 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8627c387211147ed82345656961a4525: Neither blocks nor log segments found. Creating new log.
I20260811 18:55:37.286839 10544 log.cc:824] T 00000000000000000000000000000000 P 8627c387211147ed82345656961a4525: Log is configured to *not* fsync() on all Append() calls
Aug 11 18:55:37 dist-test-slave-2j7r krb5kdc[10385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474537, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM
I20260811 18:55:37.288856 10544 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8627c387211147ed82345656961a4525: No bootstrap required, opened a new log
I20260811 18:55:37.292222 10544 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8627c387211147ed82345656961a4525 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8627c387211147ed82345656961a4525" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 37195 } }
I20260811 18:55:37.292448 10544 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8627c387211147ed82345656961a4525 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:55:37.292505 10544 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8627c387211147ed82345656961a4525 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8627c387211147ed82345656961a4525, State: Initialized, Role: FOLLOWER
I20260811 18:55:37.292783 10548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:37.280231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35590 (local address 127.27.180.190:37195)
0811 18:55:37.280659 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:37.280674 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:37.280709 (+ 35us) server_negotiation.cc:407] Connection header received
0811 18:55:37.281489 (+ 780us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:37.281511 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:37.281834 (+ 323us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:37.282136 (+ 302us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:37.282919 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:37.283887 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:37.284702 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:37.285198 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:37.285342 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:37.287902 (+ 2560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:37.287909 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:37.287922 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:37.287948 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:37.290062 (+ 2114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:37.290529 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:37.290533 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:37.290537 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:37.290637 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:37.290899 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:37.290901 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:37.290903 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:37.291261 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:37.291420 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:37.291705 (+ 285us) server_negotiation.cc:299] Negotiation successful
0811 18:55:37.291978 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288,"thread_start_us":113,"threads_started":1}
I20260811 18:55:37.292960 10544 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8627c387211147ed82345656961a4525 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8627c387211147ed82345656961a4525" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 37195 } }
I20260811 18:55:37.293118 10544 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8627c387211147ed82345656961a4525 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:55:37.293183 10544 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8627c387211147ed82345656961a4525 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:55:37.293278 10544 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8627c387211147ed82345656961a4525 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:55:37.294257 10544 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8627c387211147ed82345656961a4525 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8627c387211147ed82345656961a4525" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 37195 } }
I20260811 18:55:37.294600 10544 leader_election.cc:302] T 00000000000000000000000000000000 P 8627c387211147ed82345656961a4525 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8627c387211147ed82345656961a4525; no voters:
I20260811 18:55:37.294919 10544 leader_election.cc:288] T 00000000000000000000000000000000 P 8627c387211147ed82345656961a4525 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:55:37.295020 10551 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8627c387211147ed82345656961a4525 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:55:37.295261 10551 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8627c387211147ed82345656961a4525 [term 1 LEADER]: Becoming Leader. State: Replica: 8627c387211147ed82345656961a4525, State: Running, Role: LEADER
I20260811 18:55:37.295581 10551 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8627c387211147ed82345656961a4525 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8627c387211147ed82345656961a4525" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 37195 } }
I20260811 18:55:37.296098 10544 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8627c387211147ed82345656961a4525 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:55:37.297188 10555 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8627c387211147ed82345656961a4525 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8627c387211147ed82345656961a4525. Latest consensus state: current_term: 1 leader_uuid: "8627c387211147ed82345656961a4525" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8627c387211147ed82345656961a4525" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 37195 } } }
I20260811 18:55:37.297313 10555 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8627c387211147ed82345656961a4525 [sys.catalog]: This master's current role is: LEADER
I20260811 18:55:37.297633 10552 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8627c387211147ed82345656961a4525 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8627c387211147ed82345656961a4525" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8627c387211147ed82345656961a4525" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 37195 } } }
I20260811 18:55:37.297768 10552 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8627c387211147ed82345656961a4525 [sys.catalog]: This master's current role is: LEADER
I20260811 18:55:37.297689 10572 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:55:37.300645 10572 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:55:37.306339 10572 catalog_manager.cc:1347] Generated new cluster ID: fa4e3987a20845c4adb7e05dbc575c79
I20260811 18:55:37.306411 10572 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:55:37.327431 10572 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:55:37.328364 10572 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:55:37.339062 10572 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8627c387211147ed82345656961a4525: Generated new TSK 0
I20260811 18:55:37.339902 10572 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.27.180.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.27.180.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.27.180.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/krb5kdc/oryx_127.27.180.129.keytab.
Entry for principal oryx/127.27.180.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/krb5kdc/oryx_127.27.180.129.keytab.
I20260811 18:55:37.371822 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.180.129:0
--local_ip_for_outbound_sockets=127.27.180.129
--webserver_interface=127.27.180.129
--webserver_port=0
--tserver_master_addrs=127.27.180.190:37195
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/krb5kdc/oryx_127.27.180.129.keytab
--principal=oryx/127.27.180.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:42439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:55:37.483042 10594 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:55:37.483296 10594 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:55:37.483340 10594 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:55:37.487315 10594 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:55:37.487372 10594 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:55:37.487423 10594 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 18:55:37.487476 10594 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.180.129
I20260811 18:55:37.492632 10594 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:42439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/krb5kdc/oryx_127.27.180.129.keytab
--principal=oryx/127.27.180.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.180.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.180.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.27.180.190:37195
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10594
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.180.129
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:55:37.493656 10594 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:55:37.494635 10594 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:55:37.503880 10608 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:37.503880 10612 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:37.503921 10606 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:55:37.504086 10594 server_base.cc:1034] running on GCE node
I20260811 18:55:37.504492 10594 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:55:37.505121 10594 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:55:37.506287 10594 hybrid_clock.cc:648] HybridClock initialized: now 1786474537506271 us; error 32 us; skew 500 ppm
Aug 11 18:55:37 dist-test-slave-2j7r krb5kdc[10385](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474537, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.180.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:55:37.508946 10594 init.cc:377] Logged in from keytab as oryx/127.27.180.129@KRBTEST.COM (short username oryx)
I20260811 18:55:37.509975 10594 webserver.cc:460] Webserver started at http://127.27.180.129:33669/ using document root <none> and password file <none>
I20260811 18:55:37.510512 10594 fs_manager.cc:359] Metadata directory not provided
I20260811 18:55:37.510578 10594 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:55:37.510779 10594 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:55:37.512774 10594 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/minicluster-data/ts-0/data/instance:
uuid: "eb05f4d58b3f470881b5172e1f312e0d"
format_stamp: "Formatted at 2026-08-11 18:55:37 on dist-test-slave-2j7r"
server_key: "1ea372b2d724307c2a8c706d6618999f"
server_key_iv: "c2af9934240ec2f64d3dac229a4d6f86"
server_key_version: "encryptionkey@0"
I20260811 18:55:37.513338 10594 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/minicluster-data/ts-0/wal/instance:
uuid: "eb05f4d58b3f470881b5172e1f312e0d"
format_stamp: "Formatted at 2026-08-11 18:55:37 on dist-test-slave-2j7r"
server_key: "1ea372b2d724307c2a8c706d6618999f"
server_key_iv: "c2af9934240ec2f64d3dac229a4d6f86"
server_key_version: "encryptionkey@0"
I20260811 18:55:37.516847 10594 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.003s
I20260811 18:55:37.519205 10627 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:37.520366 10594 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260811 18:55:37.520478 10594 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/minicluster-data/ts-0/wal
uuid: "eb05f4d58b3f470881b5172e1f312e0d"
format_stamp: "Formatted at 2026-08-11 18:55:37 on dist-test-slave-2j7r"
server_key: "1ea372b2d724307c2a8c706d6618999f"
server_key_iv: "c2af9934240ec2f64d3dac229a4d6f86"
server_key_version: "encryptionkey@0"
I20260811 18:55:37.520572 10594 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:55:37.547114 10594 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:55:37.550046 10594 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:55:37.550232 10594 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:55:37.551692 10594 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:55:37.551743 10594 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:37.551796 10594 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:55:37.551826 10594 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 18:55:37 dist-test-slave-2j7r krb5kdc[10385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474537, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.180.129@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM
I20260811 18:55:37.562495 10594 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.129:38277
I20260811 18:55:37.562544 10829 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.129:38277 every 8 connection(s)
I20260811 18:55:37.563565 10594 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/minicluster-data/ts-0/data/info.pb
I20260811 18:55:37.566125 10548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:37.552483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.129:34719 (local address 127.27.180.190:37195)
0811 18:55:37.552630 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:37.552635 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:37.553525 (+ 890us) server_negotiation.cc:407] Connection header received
0811 18:55:37.554574 (+ 1049us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:37.554578 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:37.554637 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:37.554693 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:37.556347 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:37.557195 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:37.558734 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:37.559095 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:37.559213 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:37.562511 (+ 3298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:37.562515 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:37.562518 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:37.562548 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:37.563965 (+ 1417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:37.564547 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:37.564550 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:37.564552 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:37.564633 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:37.564953 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:37.564956 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:37.564958 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:37.565198 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:37.565287 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:37.565809 (+ 522us) server_negotiation.cc:299] Negotiation successful
0811 18:55:37.565960 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 18:55:37.566917 10642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:37.552886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:37195 (local address 127.27.180.129:34719)
0811 18:55:37.553358 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:37.553420 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:37.554332 (+ 912us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:37.555105 (+ 773us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:37.555113 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:37.555547 (+ 434us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:37.556171 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:37.556191 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:37.557339 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:37.557343 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:37.558608 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:37.558620 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:37.559436 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:37.559442 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:37.559634 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:37.560280 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:37.560303 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:37.562287 (+ 1984us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:55:37.564098 (+ 1811us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:37.564103 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:37.564117 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:37.564443 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:37.564717 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:37.564720 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:37.564722 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:37.564853 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:37.565297 (+ 444us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:55:37.565304 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:55:37.565578 (+ 274us) client_negotiation.cc:769] Sending connection context
0811 18:55:37.565744 (+ 166us) client_negotiation.cc:240] Negotiation successful
0811 18:55:37.566062 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":142,"threads_started":1}
I20260811 18:55:37.568091 10830 heartbeater.cc:347] Connected to a master server at 127.27.180.190:37195
I20260811 18:55:37.568233 10830 heartbeater.cc:464] Registering TS with master...
I20260811 18:55:37.568745 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 10594
I20260811 18:55:37.568763 10830 heartbeater.cc:511] Master 127.27.180.190:37195 requested a full tablet report, sending...
I20260811 18:55:37.568869 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/minicluster-data/ts-0/wal/instance
I20260811 18:55:37.569235 28370 external_mini_cluster.cc:1428] Setting key 34895898fd0e1a5600a65a474c32b3b5
I20260811 18:55:37.570586 10462 ts_manager.cc:194] Registered new tserver with Master: eb05f4d58b3f470881b5172e1f312e0d (127.27.180.129:38277)
I20260811 18:55:37.571864 10462 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.27.180.129@KRBTEST.COM'} at 127.27.180.129:34719
I20260811 18:55:37.577538 10548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:37.570402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.129:34103 (local address 127.27.180.190:37195)
0811 18:55:37.570504 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:37.570508 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:37.570737 (+ 229us) server_negotiation.cc:407] Connection header received
0811 18:55:37.570900 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:37.570915 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:37.570962 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:37.571041 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:37.571512 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:37.572169 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:37.573213 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:37.573527 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:37.573624 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:37.574772 (+ 1148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:37.574775 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:37.574777 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:37.574798 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:37.576058 (+ 1260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:37.576495 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:37.576497 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:37.576498 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:37.576562 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:37.576812 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:37.576814 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:37.576815 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:37.576960 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:37.577027 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:37.577332 (+ 305us) server_negotiation.cc:299] Negotiation successful
0811 18:55:37.577429 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 18:55:37.577615 10842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:37.570310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:37195 (local address 127.27.180.129:34103)
0811 18:55:37.570662 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:37.570676 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:37.570799 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:37.571073 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:37.571076 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:37.571279 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:37.571395 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:37.571402 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:37.572302 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:37.572305 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:37.573079 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:37.573115 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:37.573652 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:37.573655 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:37.573746 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:37.574252 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:37.574266 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:37.574636 (+ 370us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:55:37.576165 (+ 1529us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:37.576168 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:37.576170 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:37.576410 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:37.576637 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:37.576640 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:37.576642 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:37.576735 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:37.577040 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:55:37.577043 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:55:37.577170 (+ 127us) client_negotiation.cc:769] Sending connection context
0811 18:55:37.577286 (+ 116us) client_negotiation.cc:240] Negotiation successful
0811 18:55:37.577423 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":102,"threads_started":1}
WARNING: no policy specified for oryx/127.27.180.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.27.180.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.27.180.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/krb5kdc/oryx_127.27.180.130.keytab.
Entry for principal oryx/127.27.180.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/krb5kdc/oryx_127.27.180.130.keytab.
I20260811 18:55:37.596303 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.180.130:0
--local_ip_for_outbound_sockets=127.27.180.130
--webserver_interface=127.27.180.130
--webserver_port=0
--tserver_master_addrs=127.27.180.190:37195
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/krb5kdc/oryx_127.27.180.130.keytab
--principal=oryx/127.27.180.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:42439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:55:37.707422 10845 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:55:37.707827 10845 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:55:37.707926 10845 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:55:37.713737 10845 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:55:37.713795 10845 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:55:37.713837 10845 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 18:55:37.713888 10845 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.180.130
I20260811 18:55:37.719144 10845 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:42439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/krb5kdc/oryx_127.27.180.130.keytab
--principal=oryx/127.27.180.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.180.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.180.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.27.180.190:37195
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10845
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.180.130
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:55:37.720165 10845 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:55:37.721169 10845 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:55:37.730651 10853 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:55:37.730806 10845 server_base.cc:1034] running on GCE node
W20260811 18:55:37.730648 10857 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:37.730716 10852 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:55:37.731304 10845 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:55:37.731906 10845 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:55:37.733050 10845 hybrid_clock.cc:648] HybridClock initialized: now 1786474537733026 us; error 38 us; skew 500 ppm
Aug 11 18:55:37 dist-test-slave-2j7r krb5kdc[10385](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474537, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.180.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:55:37.735615 10845 init.cc:377] Logged in from keytab as oryx/127.27.180.130@KRBTEST.COM (short username oryx)
I20260811 18:55:37.736469 10845 webserver.cc:460] Webserver started at http://127.27.180.130:42261/ using document root <none> and password file <none>
I20260811 18:55:37.736994 10845 fs_manager.cc:359] Metadata directory not provided
I20260811 18:55:37.737052 10845 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:55:37.737282 10845 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:55:37.739310 10845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/minicluster-data/ts-1/data/instance:
uuid: "1c9d229dc7694099b581b5c31563108d"
format_stamp: "Formatted at 2026-08-11 18:55:37 on dist-test-slave-2j7r"
server_key: "5963a84abc0683da7a46789c5571941c"
server_key_iv: "d0e22a4d857fc029f163f15a6a00845e"
server_key_version: "encryptionkey@0"
I20260811 18:55:37.739840 10845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/minicluster-data/ts-1/wal/instance:
uuid: "1c9d229dc7694099b581b5c31563108d"
format_stamp: "Formatted at 2026-08-11 18:55:37 on dist-test-slave-2j7r"
server_key: "5963a84abc0683da7a46789c5571941c"
server_key_iv: "d0e22a4d857fc029f163f15a6a00845e"
server_key_version: "encryptionkey@0"
I20260811 18:55:37.743433 10845 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 18:55:37.745748 10874 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:37.746975 10845 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 18:55:37.747097 10845 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/minicluster-data/ts-1/wal
uuid: "1c9d229dc7694099b581b5c31563108d"
format_stamp: "Formatted at 2026-08-11 18:55:37 on dist-test-slave-2j7r"
server_key: "5963a84abc0683da7a46789c5571941c"
server_key_iv: "d0e22a4d857fc029f163f15a6a00845e"
server_key_version: "encryptionkey@0"
I20260811 18:55:37.747191 10845 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:55:37.760857 10845 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:55:37.763924 10845 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:55:37.764104 10845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:55:37.765657 10845 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:55:37.765713 10845 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:37.765769 10845 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:55:37.765793 10845 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 18:55:37 dist-test-slave-2j7r krb5kdc[10385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474537, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.180.130@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM
I20260811 18:55:37.776399 10845 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.130:37907
I20260811 18:55:37.776487 11078 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.130:37907 every 8 connection(s)
I20260811 18:55:37.777479 10845 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/minicluster-data/ts-1/data/info.pb
I20260811 18:55:37.778863 10548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:37.766380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.130:50965 (local address 127.27.180.190:37195)
0811 18:55:37.766513 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:37.766517 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:37.767609 (+ 1092us) server_negotiation.cc:407] Connection header received
0811 18:55:37.768695 (+ 1086us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:37.768699 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:37.768761 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:37.768821 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:37.770199 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:37.770776 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:37.772004 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:37.772222 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:37.772313 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:37.775210 (+ 2897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:37.775214 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:37.775217 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:37.775246 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:37.776625 (+ 1379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:37.777176 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:37.777178 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:37.777179 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:37.777251 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:37.777642 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:37.777644 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:37.777645 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:37.777847 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:37.777951 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:37.778642 (+ 691us) server_negotiation.cc:299] Negotiation successful
0811 18:55:37.778742 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 18:55:37.779460 10888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:37.766953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:37195 (local address 127.27.180.130:50965)
0811 18:55:37.767434 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:37.767477 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:37.768388 (+ 911us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:37.769227 (+ 839us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:37.769234 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:37.769562 (+ 328us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:37.770045 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:37.770065 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:37.770904 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:37.770908 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:37.771879 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:37.771890 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:37.772413 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:37.772418 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:37.772643 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:37.773231 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:37.773253 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:37.775026 (+ 1773us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:55:37.776727 (+ 1701us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:37.776733 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:37.776745 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:37.777064 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:37.777328 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:37.777331 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:37.777333 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:37.777509 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:37.777961 (+ 452us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:55:37.777968 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:55:37.778254 (+ 286us) client_negotiation.cc:769] Sending connection context
0811 18:55:37.778449 (+ 195us) client_negotiation.cc:240] Negotiation successful
0811 18:55:37.778708 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":132,"threads_started":1}
I20260811 18:55:37.780534 11079 heartbeater.cc:347] Connected to a master server at 127.27.180.190:37195
I20260811 18:55:37.780670 11079 heartbeater.cc:464] Registering TS with master...
I20260811 18:55:37.781153 11079 heartbeater.cc:511] Master 127.27.180.190:37195 requested a full tablet report, sending...
I20260811 18:55:37.782142 10461 ts_manager.cc:194] Registered new tserver with Master: 1c9d229dc7694099b581b5c31563108d (127.27.180.130:37907)
I20260811 18:55:37.782748 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 10845
I20260811 18:55:37.782871 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/minicluster-data/ts-1/wal/instance
I20260811 18:55:37.782848 10461 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.27.180.130@KRBTEST.COM'} at 127.27.180.130:50965
I20260811 18:55:37.783160 28370 external_mini_cluster.cc:1428] Setting key 73498260962ca9f0506c52b67f5bbe36
I20260811 18:55:37.789855 10548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:37.783533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.130:55977 (local address 127.27.180.190:37195)
0811 18:55:37.783650 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:37.783654 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:37.783666 (+ 12us) server_negotiation.cc:407] Connection header received
0811 18:55:37.783697 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:37.783699 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:37.783732 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:37.783810 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:37.784425 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:37.785145 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:37.785886 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:37.786184 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:37.786280 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:37.787177 (+ 897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:37.787180 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:37.787182 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:37.787202 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:37.788414 (+ 1212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:37.788773 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:37.788776 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:37.788778 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:37.788845 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:37.789096 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:37.789100 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:37.789101 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:37.789301 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:37.789395 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:37.789623 (+ 228us) server_negotiation.cc:299] Negotiation successful
0811 18:55:37.789719 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 18:55:37.789870 11087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:37.782483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:37195 (local address 127.27.180.130:55977)
0811 18:55:37.782765 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:37.782778 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:37.782859 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:37.783938 (+ 1079us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:37.783941 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:37.784182 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:37.784309 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:37.784316 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:37.785251 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:37.785254 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:37.785784 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:37.785791 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:37.786276 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:37.786278 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:37.786337 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:37.786743 (+ 406us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:37.786755 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:37.787054 (+ 299us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:55:37.788510 (+ 1456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:37.788513 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:37.788515 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:37.788691 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:37.788927 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:37.788930 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:37.788932 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:37.789006 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:37.789390 (+ 384us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:55:37.789393 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:55:37.789479 (+ 86us) client_negotiation.cc:769] Sending connection context
0811 18:55:37.789564 (+ 85us) client_negotiation.cc:240] Negotiation successful
0811 18:55:37.789692 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":109,"threads_started":1}
WARNING: no policy specified for oryx/127.27.180.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.27.180.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.27.180.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/krb5kdc/oryx_127.27.180.131.keytab.
Entry for principal oryx/127.27.180.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/krb5kdc/oryx_127.27.180.131.keytab.
I20260811 18:55:37.809491 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.180.131:0
--local_ip_for_outbound_sockets=127.27.180.131
--webserver_interface=127.27.180.131
--webserver_port=0
--tserver_master_addrs=127.27.180.190:37195
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/krb5kdc/oryx_127.27.180.131.keytab
--principal=oryx/127.27.180.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:42439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:55:37.922528 11091 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:55:37.922780 11091 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:55:37.922824 11091 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:55:37.927004 11091 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:55:37.927071 11091 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:55:37.927115 11091 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 18:55:37.927147 11091 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.180.131
I20260811 18:55:37.932397 11091 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:42439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/krb5kdc/oryx_127.27.180.131.keytab
--principal=oryx/127.27.180.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.180.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.180.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.27.180.190:37195
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11091
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.180.131
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:55:37.933470 11091 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:55:37.934521 11091 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:55:37.944332 11098 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:55:37.944386 11091 server_base.cc:1034] running on GCE node
W20260811 18:55:37.944340 11097 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:37.944346 11103 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:55:37.944978 11091 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:55:37.945669 11091 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:55:37.946834 11091 hybrid_clock.cc:648] HybridClock initialized: now 1786474537946806 us; error 36 us; skew 500 ppm
Aug 11 18:55:37 dist-test-slave-2j7r krb5kdc[10385](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474537, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.180.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:55:37.949724 11091 init.cc:377] Logged in from keytab as oryx/127.27.180.131@KRBTEST.COM (short username oryx)
I20260811 18:55:37.950659 11091 webserver.cc:460] Webserver started at http://127.27.180.131:41957/ using document root <none> and password file <none>
I20260811 18:55:37.951224 11091 fs_manager.cc:359] Metadata directory not provided
I20260811 18:55:37.951292 11091 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:55:37.951486 11091 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:55:37.953583 11091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/minicluster-data/ts-2/data/instance:
uuid: "7205a2fa560f4b448715b67be73c1174"
format_stamp: "Formatted at 2026-08-11 18:55:37 on dist-test-slave-2j7r"
server_key: "4b966c7515378e6da3f5901584f97484"
server_key_iv: "c72d7aa5efdd8e104943f6aab290095c"
server_key_version: "encryptionkey@0"
I20260811 18:55:37.954160 11091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/minicluster-data/ts-2/wal/instance:
uuid: "7205a2fa560f4b448715b67be73c1174"
format_stamp: "Formatted at 2026-08-11 18:55:37 on dist-test-slave-2j7r"
server_key: "4b966c7515378e6da3f5901584f97484"
server_key_iv: "c72d7aa5efdd8e104943f6aab290095c"
server_key_version: "encryptionkey@0"
I20260811 18:55:37.957871 11091 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:55:37.960297 11117 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:37.961619 11091 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 18:55:37.961740 11091 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/minicluster-data/ts-2/wal
uuid: "7205a2fa560f4b448715b67be73c1174"
format_stamp: "Formatted at 2026-08-11 18:55:37 on dist-test-slave-2j7r"
server_key: "4b966c7515378e6da3f5901584f97484"
server_key_iv: "c72d7aa5efdd8e104943f6aab290095c"
server_key_version: "encryptionkey@0"
I20260811 18:55:37.961855 11091 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:55:37.974993 11091 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:55:37.977695 11091 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:55:37.977843 11091 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:55:37.979229 11091 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:55:37.979281 11091 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:37.979321 11091 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:55:37.979337 11091 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 18:55:37 dist-test-slave-2j7r krb5kdc[10385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474537, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.180.131@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM
I20260811 18:55:37.989825 11091 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.131:43411
I20260811 18:55:37.989876 11317 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.131:43411 every 8 connection(s)
I20260811 18:55:37.990875 11091 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/minicluster-data/ts-2/data/info.pb
I20260811 18:55:37.992019 10548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:37.979929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.131:45583 (local address 127.27.180.190:37195)
0811 18:55:37.980076 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:37.980082 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:37.980975 (+ 893us) server_negotiation.cc:407] Connection header received
0811 18:55:37.982027 (+ 1052us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:37.982031 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:37.982094 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:37.982194 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:37.983510 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:37.984047 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:37.985282 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:37.985587 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:37.985685 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:37.988329 (+ 2644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:37.988332 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:37.988334 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:37.988362 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:37.989904 (+ 1542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:37.990469 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:37.990472 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:37.990473 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:37.990554 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:37.990894 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:37.990896 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:37.990897 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:37.991083 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:37.991160 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:37.991795 (+ 635us) server_negotiation.cc:299] Negotiation successful
0811 18:55:37.991899 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 18:55:37.992765 11134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:37.980320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:37195 (local address 127.27.180.131:45583)
0811 18:55:37.980818 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:37.980868 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:37.981738 (+ 870us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:37.982384 (+ 646us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:37.982394 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:37.982825 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:37.983361 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:37.983384 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:37.984172 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:37.984176 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:37.985162 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:37.985175 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:37.985741 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:37.985746 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:37.985911 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:37.986485 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:37.986507 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:37.988164 (+ 1657us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:55:37.990034 (+ 1870us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:37.990040 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:37.990052 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:37.990360 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:37.990645 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:37.990648 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:37.990650 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:37.990813 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:37.991182 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:55:37.991190 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:55:37.991520 (+ 330us) client_negotiation.cc:769] Sending connection context
0811 18:55:37.991716 (+ 196us) client_negotiation.cc:240] Negotiation successful
0811 18:55:37.992019 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":114,"threads_started":1}
I20260811 18:55:37.994030 11320 heartbeater.cc:347] Connected to a master server at 127.27.180.190:37195
I20260811 18:55:37.994168 11320 heartbeater.cc:464] Registering TS with master...
I20260811 18:55:37.994683 11320 heartbeater.cc:511] Master 127.27.180.190:37195 requested a full tablet report, sending...
I20260811 18:55:37.995507 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 11091
I20260811 18:55:37.995664 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/minicluster-data/ts-2/wal/instance
I20260811 18:55:37.995764 10462 ts_manager.cc:194] Registered new tserver with Master: 7205a2fa560f4b448715b67be73c1174 (127.27.180.131:43411)
I20260811 18:55:37.995996 28370 external_mini_cluster.cc:1428] Setting key 61bc465f3f1da44789dfba3faed35eae
I20260811 18:55:37.996537 10462 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.27.180.131@KRBTEST.COM'} at 127.27.180.131:45583
I20260811 18:55:37.998250 28370 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 18:55:38.002748 10548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:37.996318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.131:45341 (local address 127.27.180.190:37195)
0811 18:55:37.996440 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:37.996444 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:37.996456 (+ 12us) server_negotiation.cc:407] Connection header received
0811 18:55:37.996491 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:37.996494 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:37.996582 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:37.996672 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:37.997300 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:37.997874 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:37.998741 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:37.998964 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:37.999047 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:38.000019 (+ 972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:38.000023 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:38.000026 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:38.000050 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:38.001323 (+ 1273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:38.001694 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:38.001697 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:38.001699 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:38.001770 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:38.002019 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:38.002023 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:38.002024 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:38.002179 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:38.002264 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:38.002506 (+ 242us) server_negotiation.cc:299] Negotiation successful
0811 18:55:38.002605 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 18:55:38.002758 11336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:37.996001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:37195 (local address 127.27.180.131:45341)
0811 18:55:37.996316 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:37.996331 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:37.996427 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:37.996816 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:37.996819 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:37.997028 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:37.997168 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:37.997176 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:37.998018 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:37.998021 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:37.998631 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:37.998640 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:37.999062 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:37.999065 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:37.999154 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:37.999574 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:37.999585 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:37.999884 (+ 299us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:55:38.001420 (+ 1536us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:38.001423 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:38.001425 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:38.001609 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:38.001850 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:38.001853 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:38.001855 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:38.001941 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:38.002268 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:55:38.002271 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:55:38.002356 (+ 85us) client_negotiation.cc:769] Sending connection context
0811 18:55:38.002443 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 18:55:38.002573 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":133,"threads_started":1}
Aug 11 18:55:38 dist-test-slave-2j7r krb5kdc[10385](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474538, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 18:55:38 dist-test-slave-2j7r krb5kdc[10385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:55:38 dist-test-slave-2j7r krb5kdc[10385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
W20260811 18:55:38.022104 10548 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:55:38.016300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35606 (local address 127.27.180.190:37195)
0811 18:55:38.016424 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:38.016427 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:38.016438 (+ 11us) server_negotiation.cc:407] Connection header received
0811 18:55:38.016467 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:38.016469 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:38.016501 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:38.016579 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:38.017257 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:38.017743 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:38.018568 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:38.018786 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:38.018879 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:38.021965 (+ 3086us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35606: BlockingRecv error: recv got EOF from 127.0.0.1:35606 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 11 18:55:38 dist-test-slave-2j7r krb5kdc[10385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474538, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM
I20260811 18:55:38.031020 10548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:38.023542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35620 (local address 127.27.180.190:37195)
0811 18:55:38.023721 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:38.023725 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:38.023742 (+ 17us) server_negotiation.cc:407] Connection header received
0811 18:55:38.023907 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:38.023910 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:38.023947 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:38.024010 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:38.024610 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:38.025142 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:38.025910 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:38.026127 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:38.026215 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:38.028285 (+ 2070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:38.028288 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:38.028290 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:38.028312 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:38.029562 (+ 1250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:38.029930 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:38.029934 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:38.029935 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:38.030006 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:38.030252 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:38.030256 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:38.030257 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:38.030422 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:38.030539 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:38.030764 (+ 225us) server_negotiation.cc:299] Negotiation successful
0811 18:55:38.030875 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260811 18:55:38.033788 10462 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35620:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:55:38.036697 10462 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:55:38.050532 11366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:38.044797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.131:43411 (local address 127.0.0.1:37328)
0811 18:55:38.045339 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:38.045386 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:38.045521 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:38.046397 (+ 876us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:38.046403 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:38.046425 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:55:38.046551 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:38.046557 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:38.047724 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:38.047728 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:38.048927 (+ 1199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:38.048938 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:38.049995 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:38.049999 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:38.050102 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:38.050147 (+ 45us) client_negotiation.cc:769] Sending connection context
0811 18:55:38.050252 (+ 105us) client_negotiation.cc:240] Negotiation successful
0811 18:55:38.050373 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":83,"threads_started":1}
I20260811 18:55:38.050575 11367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:38.044971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37328 (local address 127.27.180.131:43411)
0811 18:55:38.045795 (+ 824us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:38.045807 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:38.045831 (+ 24us) server_negotiation.cc:407] Connection header received
0811 18:55:38.045933 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:38.045938 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:38.046200 (+ 262us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:38.046392 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:55:38.046660 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:38.047622 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:38.049070 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:38.049873 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:38.050010 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:38.050198 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:38.050285 (+ 87us) server_negotiation.cc:299] Negotiation successful
0811 18:55:38.050372 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":720,"thread_start_us":126,"threads_started":1}
I20260811 18:55:38.050928 11365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:38.044942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.130:37907 (local address 127.0.0.1:57478)
0811 18:55:38.045766 (+ 824us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:38.045778 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:38.045856 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:38.046822 (+ 966us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:38.046825 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:38.046846 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:55:38.046973 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:38.046979 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:38.048449 (+ 1470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:38.048452 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:38.049501 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:38.049512 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:38.050555 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:38.050559 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:38.050655 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:38.050680 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 18:55:38.050730 (+ 50us) client_negotiation.cc:240] Negotiation successful
0811 18:55:38.050792 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":763,"mutex_wait_us":1,"thread_start_us":108,"threads_started":1}
I20260811 18:55:38.051085 11370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:38.045970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38594 (local address 127.27.180.129:38277)
0811 18:55:38.046328 (+ 358us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:38.046347 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:38.046370 (+ 23us) server_negotiation.cc:407] Connection header received
0811 18:55:38.046446 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:38.046450 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:38.046625 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:38.046836 (+ 211us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:55:38.046978 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:38.048101 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:38.049496 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:38.050104 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:38.050658 (+ 554us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:38.050798 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:38.050847 (+ 49us) server_negotiation.cc:299] Negotiation successful
0811 18:55:38.050948 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":270,"thread_start_us":73,"threads_started":1}
I20260811 18:55:38.050551 11369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:38.045526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.129:38277 (local address 127.0.0.1:38594)
0811 18:55:38.046042 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:38.046054 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:38.046129 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:38.046771 (+ 642us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:38.046773 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:38.046792 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:55:38.046878 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:38.046885 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:38.048225 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:38.048229 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:38.049372 (+ 1143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:38.049382 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:38.050238 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:38.050242 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:38.050331 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:38.050352 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 18:55:38.050400 (+ 48us) client_negotiation.cc:240] Negotiation successful
0811 18:55:38.050454 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":462,"mutex_wait_us":20,"thread_start_us":91,"threads_started":1}
I20260811 18:55:38.052026 11368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:38.045479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57478 (local address 127.27.180.130:37907)
0811 18:55:38.046365 (+ 886us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:38.046375 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:38.046394 (+ 19us) server_negotiation.cc:407] Connection header received
0811 18:55:38.046474 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:38.046478 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:38.046625 (+ 147us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:38.047126 (+ 501us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:55:38.047271 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:38.048308 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:38.049623 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:38.050423 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:38.051634 (+ 1211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:38.051758 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:38.051803 (+ 45us) server_negotiation.cc:299] Negotiation successful
0811 18:55:38.051886 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":796,"thread_start_us":117,"threads_started":1}
I20260811 18:55:38.053984 11209 tablet_service.cc:1467] Processing CreateTablet for tablet d2d8f836c9e547b7a5fa0ba6a1111d0c (DEFAULT_TABLE table=test-table [id=b398c77b16b74162ac8d38f6399bc082]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:55:38.053984 10721 tablet_service.cc:1467] Processing CreateTablet for tablet d2d8f836c9e547b7a5fa0ba6a1111d0c (DEFAULT_TABLE table=test-table [id=b398c77b16b74162ac8d38f6399bc082]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:55:38.054453 10958 tablet_service.cc:1467] Processing CreateTablet for tablet d2d8f836c9e547b7a5fa0ba6a1111d0c (DEFAULT_TABLE table=test-table [id=b398c77b16b74162ac8d38f6399bc082]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:55:38.055137 10721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d2d8f836c9e547b7a5fa0ba6a1111d0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:55:38.055150 11209 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d2d8f836c9e547b7a5fa0ba6a1111d0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:55:38.055389 10958 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d2d8f836c9e547b7a5fa0ba6a1111d0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:55:38.062071 11371 tablet_bootstrap.cc:492] T d2d8f836c9e547b7a5fa0ba6a1111d0c P 7205a2fa560f4b448715b67be73c1174: Bootstrap starting.
I20260811 18:55:38.063467 11372 tablet_bootstrap.cc:492] T d2d8f836c9e547b7a5fa0ba6a1111d0c P 1c9d229dc7694099b581b5c31563108d: Bootstrap starting.
I20260811 18:55:38.064436 11371 tablet_bootstrap.cc:654] T d2d8f836c9e547b7a5fa0ba6a1111d0c P 7205a2fa560f4b448715b67be73c1174: Neither blocks nor log segments found. Creating new log.
I20260811 18:55:38.065266 11373 tablet_bootstrap.cc:492] T d2d8f836c9e547b7a5fa0ba6a1111d0c P eb05f4d58b3f470881b5172e1f312e0d: Bootstrap starting.
I20260811 18:55:38.065347 11371 log.cc:824] T d2d8f836c9e547b7a5fa0ba6a1111d0c P 7205a2fa560f4b448715b67be73c1174: Log is configured to *not* fsync() on all Append() calls
I20260811 18:55:38.065858 11372 tablet_bootstrap.cc:654] T d2d8f836c9e547b7a5fa0ba6a1111d0c P 1c9d229dc7694099b581b5c31563108d: Neither blocks nor log segments found. Creating new log.
I20260811 18:55:38.066545 11372 log.cc:824] T d2d8f836c9e547b7a5fa0ba6a1111d0c P 1c9d229dc7694099b581b5c31563108d: Log is configured to *not* fsync() on all Append() calls
I20260811 18:55:38.067142 11373 tablet_bootstrap.cc:654] T d2d8f836c9e547b7a5fa0ba6a1111d0c P eb05f4d58b3f470881b5172e1f312e0d: Neither blocks nor log segments found. Creating new log.
I20260811 18:55:38.068056 11373 log.cc:824] T d2d8f836c9e547b7a5fa0ba6a1111d0c P eb05f4d58b3f470881b5172e1f312e0d: Log is configured to *not* fsync() on all Append() calls
I20260811 18:55:38.068557 11371 tablet_bootstrap.cc:492] T d2d8f836c9e547b7a5fa0ba6a1111d0c P 7205a2fa560f4b448715b67be73c1174: No bootstrap required, opened a new log
I20260811 18:55:38.068557 11372 tablet_bootstrap.cc:492] T d2d8f836c9e547b7a5fa0ba6a1111d0c P 1c9d229dc7694099b581b5c31563108d: No bootstrap required, opened a new log
I20260811 18:55:38.068747 11372 ts_tablet_manager.cc:1397] T d2d8f836c9e547b7a5fa0ba6a1111d0c P 1c9d229dc7694099b581b5c31563108d: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260811 18:55:38.068747 11371 ts_tablet_manager.cc:1397] T d2d8f836c9e547b7a5fa0ba6a1111d0c P 7205a2fa560f4b448715b67be73c1174: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 18:55:38.069767 11373 tablet_bootstrap.cc:492] T d2d8f836c9e547b7a5fa0ba6a1111d0c P eb05f4d58b3f470881b5172e1f312e0d: No bootstrap required, opened a new log
I20260811 18:55:38.069943 11373 ts_tablet_manager.cc:1397] T d2d8f836c9e547b7a5fa0ba6a1111d0c P eb05f4d58b3f470881b5172e1f312e0d: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 18:55:38.072623 11373 raft_consensus.cc:348] T d2d8f836c9e547b7a5fa0ba6a1111d0c P eb05f4d58b3f470881b5172e1f312e0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7205a2fa560f4b448715b67be73c1174" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 43411 } } peers { permanent_uuid: "1c9d229dc7694099b581b5c31563108d" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 37907 } } peers { permanent_uuid: "eb05f4d58b3f470881b5172e1f312e0d" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38277 } }
I20260811 18:55:38.072832 11373 raft_consensus.cc:374] T d2d8f836c9e547b7a5fa0ba6a1111d0c P eb05f4d58b3f470881b5172e1f312e0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:55:38.072679 11371 raft_consensus.cc:348] T d2d8f836c9e547b7a5fa0ba6a1111d0c P 7205a2fa560f4b448715b67be73c1174 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7205a2fa560f4b448715b67be73c1174" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 43411 } } peers { permanent_uuid: "1c9d229dc7694099b581b5c31563108d" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 37907 } } peers { permanent_uuid: "eb05f4d58b3f470881b5172e1f312e0d" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38277 } }
I20260811 18:55:38.072679 11372 raft_consensus.cc:348] T d2d8f836c9e547b7a5fa0ba6a1111d0c P 1c9d229dc7694099b581b5c31563108d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7205a2fa560f4b448715b67be73c1174" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 43411 } } peers { permanent_uuid: "1c9d229dc7694099b581b5c31563108d" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 37907 } } peers { permanent_uuid: "eb05f4d58b3f470881b5172e1f312e0d" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38277 } }
I20260811 18:55:38.072902 11373 raft_consensus.cc:729] T d2d8f836c9e547b7a5fa0ba6a1111d0c P eb05f4d58b3f470881b5172e1f312e0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb05f4d58b3f470881b5172e1f312e0d, State: Initialized, Role: FOLLOWER
I20260811 18:55:38.072925 11372 raft_consensus.cc:374] T d2d8f836c9e547b7a5fa0ba6a1111d0c P 1c9d229dc7694099b581b5c31563108d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:55:38.072952 11371 raft_consensus.cc:374] T d2d8f836c9e547b7a5fa0ba6a1111d0c P 7205a2fa560f4b448715b67be73c1174 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:55:38.072997 11371 raft_consensus.cc:729] T d2d8f836c9e547b7a5fa0ba6a1111d0c P 7205a2fa560f4b448715b67be73c1174 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7205a2fa560f4b448715b67be73c1174, State: Initialized, Role: FOLLOWER
I20260811 18:55:38.073017 11372 raft_consensus.cc:729] T d2d8f836c9e547b7a5fa0ba6a1111d0c P 1c9d229dc7694099b581b5c31563108d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c9d229dc7694099b581b5c31563108d, State: Initialized, Role: FOLLOWER
I20260811 18:55:38.073375 11373 consensus_queue.cc:260] T d2d8f836c9e547b7a5fa0ba6a1111d0c P eb05f4d58b3f470881b5172e1f312e0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7205a2fa560f4b448715b67be73c1174" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 43411 } } peers { permanent_uuid: "1c9d229dc7694099b581b5c31563108d" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 37907 } } peers { permanent_uuid: "eb05f4d58b3f470881b5172e1f312e0d" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38277 } }
I20260811 18:55:38.073462 11372 consensus_queue.cc:260] T d2d8f836c9e547b7a5fa0ba6a1111d0c P 1c9d229dc7694099b581b5c31563108d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7205a2fa560f4b448715b67be73c1174" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 43411 } } peers { permanent_uuid: "1c9d229dc7694099b581b5c31563108d" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 37907 } } peers { permanent_uuid: "eb05f4d58b3f470881b5172e1f312e0d" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38277 } }
I20260811 18:55:38.073473 11371 consensus_queue.cc:260] T d2d8f836c9e547b7a5fa0ba6a1111d0c P 7205a2fa560f4b448715b67be73c1174 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7205a2fa560f4b448715b67be73c1174" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 43411 } } peers { permanent_uuid: "1c9d229dc7694099b581b5c31563108d" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 37907 } } peers { permanent_uuid: "eb05f4d58b3f470881b5172e1f312e0d" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38277 } }
I20260811 18:55:38.074026 10830 heartbeater.cc:503] Master 127.27.180.190:37195 was elected leader, sending a full tablet report...
I20260811 18:55:38.074390 11373 ts_tablet_manager.cc:1428] T d2d8f836c9e547b7a5fa0ba6a1111d0c P eb05f4d58b3f470881b5172e1f312e0d: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20260811 18:55:38.074402 11372 ts_tablet_manager.cc:1428] T d2d8f836c9e547b7a5fa0ba6a1111d0c P 1c9d229dc7694099b581b5c31563108d: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 18:55:38.074412 11371 ts_tablet_manager.cc:1428] T d2d8f836c9e547b7a5fa0ba6a1111d0c P 7205a2fa560f4b448715b67be73c1174: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260811 18:55:38.074719 11079 heartbeater.cc:503] Master 127.27.180.190:37195 was elected leader, sending a full tablet report...
I20260811 18:55:38.074719 11320 heartbeater.cc:503] Master 127.27.180.190:37195 was elected leader, sending a full tablet report...
W20260811 18:55:38.241922 11322 tablet.cc:2369] T d2d8f836c9e547b7a5fa0ba6a1111d0c P 7205a2fa560f4b448715b67be73c1174: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 18:55:38.278822 11080 tablet.cc:2369] T d2d8f836c9e547b7a5fa0ba6a1111d0c P 1c9d229dc7694099b581b5c31563108d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:55:38.295042 11381 raft_consensus.cc:482] T d2d8f836c9e547b7a5fa0ba6a1111d0c P eb05f4d58b3f470881b5172e1f312e0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:55:38.295265 11381 raft_consensus.cc:504] T d2d8f836c9e547b7a5fa0ba6a1111d0c P eb05f4d58b3f470881b5172e1f312e0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7205a2fa560f4b448715b67be73c1174" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 43411 } } peers { permanent_uuid: "1c9d229dc7694099b581b5c31563108d" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 37907 } } peers { permanent_uuid: "eb05f4d58b3f470881b5172e1f312e0d" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38277 } }
I20260811 18:55:38.296542 11381 leader_election.cc:288] T d2d8f836c9e547b7a5fa0ba6a1111d0c P eb05f4d58b3f470881b5172e1f312e0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7205a2fa560f4b448715b67be73c1174 (127.27.180.131:43411), 1c9d229dc7694099b581b5c31563108d (127.27.180.130:37907)
I20260811 18:55:38.301723 11368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:38.296888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.129:32895 (local address 127.27.180.130:37907)
0811 18:55:38.297012 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:38.297017 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:38.297216 (+ 199us) server_negotiation.cc:407] Connection header received
0811 18:55:38.297350 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:38.297353 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:38.297396 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:38.297478 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:55:38.297867 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:38.298683 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:38.300342 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:38.300979 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:38.301114 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:38.301171 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:38.301488 (+ 317us) server_negotiation.cc:299] Negotiation successful
0811 18:55:38.301584 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 18:55:38.301718 11388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:38.296754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.131:43411 (local address 127.27.180.129:45389)
0811 18:55:38.297058 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:38.297073 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:38.297233 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:38.297498 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:38.297501 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:38.297543 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:55:38.297755 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:38.297760 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:38.298788 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:38.298792 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:38.300182 (+ 1390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:38.300194 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:38.301116 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:38.301120 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:38.301287 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:38.301350 (+ 63us) client_negotiation.cc:769] Sending connection context
0811 18:55:38.301428 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 18:55:38.301532 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"mutex_wait_us":194,"spinlock_wait_cycles":32640,"thread_start_us":73,"threads_started":1}
I20260811 18:55:38.301730 11389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:38.296800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.130:37907 (local address 127.27.180.129:32895)
0811 18:55:38.297145 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:38.297156 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:38.297266 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:38.297498 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:38.297501 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:38.297545 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:55:38.297749 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:38.297757 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:38.298789 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:38.298792 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:38.300182 (+ 1390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:38.300194 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:38.301119 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:38.301122 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:38.301301 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:38.301359 (+ 58us) client_negotiation.cc:769] Sending connection context
0811 18:55:38.301438 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 18:55:38.301532 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"mutex_wait_us":202,"spinlock_wait_cycles":8576,"thread_start_us":73,"threads_started":1}
I20260811 18:55:38.302337 10996 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2d8f836c9e547b7a5fa0ba6a1111d0c" candidate_uuid: "eb05f4d58b3f470881b5172e1f312e0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c9d229dc7694099b581b5c31563108d" is_pre_election: true
I20260811 18:55:38.302448 11367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:38.296860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.129:45389 (local address 127.27.180.131:43411)
0811 18:55:38.297044 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:38.297048 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:38.297183 (+ 135us) server_negotiation.cc:407] Connection header received
0811 18:55:38.297338 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:38.297341 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:38.297403 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:38.297477 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:55:38.297867 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:38.298687 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:38.300342 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:38.300958 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:38.301114 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:38.301171 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:38.302273 (+ 1102us) server_negotiation.cc:299] Negotiation successful
0811 18:55:38.302352 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260811 18:55:38.302625 10996 raft_consensus.cc:2454] T d2d8f836c9e547b7a5fa0ba6a1111d0c P 1c9d229dc7694099b581b5c31563108d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb05f4d58b3f470881b5172e1f312e0d in term 0.
I20260811 18:55:38.303007 11238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2d8f836c9e547b7a5fa0ba6a1111d0c" candidate_uuid: "eb05f4d58b3f470881b5172e1f312e0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7205a2fa560f4b448715b67be73c1174" is_pre_election: true
I20260811 18:55:38.303117 10635 leader_election.cc:302] T d2d8f836c9e547b7a5fa0ba6a1111d0c P eb05f4d58b3f470881b5172e1f312e0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c9d229dc7694099b581b5c31563108d, eb05f4d58b3f470881b5172e1f312e0d; no voters:
I20260811 18:55:38.303304 11238 raft_consensus.cc:2454] T d2d8f836c9e547b7a5fa0ba6a1111d0c P 7205a2fa560f4b448715b67be73c1174 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb05f4d58b3f470881b5172e1f312e0d in term 0.
I20260811 18:55:38.303390 11381 raft_consensus.cc:2781] T d2d8f836c9e547b7a5fa0ba6a1111d0c P eb05f4d58b3f470881b5172e1f312e0d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:55:38.303480 11381 raft_consensus.cc:482] T d2d8f836c9e547b7a5fa0ba6a1111d0c P eb05f4d58b3f470881b5172e1f312e0d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:55:38.303531 11381 raft_consensus.cc:3037] T d2d8f836c9e547b7a5fa0ba6a1111d0c P eb05f4d58b3f470881b5172e1f312e0d [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:55:38.304589 11381 raft_consensus.cc:504] T d2d8f836c9e547b7a5fa0ba6a1111d0c P eb05f4d58b3f470881b5172e1f312e0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7205a2fa560f4b448715b67be73c1174" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 43411 } } peers { permanent_uuid: "1c9d229dc7694099b581b5c31563108d" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 37907 } } peers { permanent_uuid: "eb05f4d58b3f470881b5172e1f312e0d" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38277 } }
I20260811 18:55:38.304943 11381 leader_election.cc:288] T d2d8f836c9e547b7a5fa0ba6a1111d0c P eb05f4d58b3f470881b5172e1f312e0d [CANDIDATE]: Term 1 election: Requested vote from peers 7205a2fa560f4b448715b67be73c1174 (127.27.180.131:43411), 1c9d229dc7694099b581b5c31563108d (127.27.180.130:37907)
I20260811 18:55:38.305269 11238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2d8f836c9e547b7a5fa0ba6a1111d0c" candidate_uuid: "eb05f4d58b3f470881b5172e1f312e0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7205a2fa560f4b448715b67be73c1174"
I20260811 18:55:38.305294 10996 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2d8f836c9e547b7a5fa0ba6a1111d0c" candidate_uuid: "eb05f4d58b3f470881b5172e1f312e0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c9d229dc7694099b581b5c31563108d"
I20260811 18:55:38.305375 11238 raft_consensus.cc:3037] T d2d8f836c9e547b7a5fa0ba6a1111d0c P 7205a2fa560f4b448715b67be73c1174 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:55:38.305397 10996 raft_consensus.cc:3037] T d2d8f836c9e547b7a5fa0ba6a1111d0c P 1c9d229dc7694099b581b5c31563108d [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:55:38.306339 10996 raft_consensus.cc:2454] T d2d8f836c9e547b7a5fa0ba6a1111d0c P 1c9d229dc7694099b581b5c31563108d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb05f4d58b3f470881b5172e1f312e0d in term 1.
I20260811 18:55:38.306602 10635 leader_election.cc:302] T d2d8f836c9e547b7a5fa0ba6a1111d0c P eb05f4d58b3f470881b5172e1f312e0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c9d229dc7694099b581b5c31563108d, eb05f4d58b3f470881b5172e1f312e0d; no voters:
I20260811 18:55:38.306710 11238 raft_consensus.cc:2454] T d2d8f836c9e547b7a5fa0ba6a1111d0c P 7205a2fa560f4b448715b67be73c1174 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb05f4d58b3f470881b5172e1f312e0d in term 1.
I20260811 18:55:38.306761 11381 raft_consensus.cc:2781] T d2d8f836c9e547b7a5fa0ba6a1111d0c P eb05f4d58b3f470881b5172e1f312e0d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:55:38.306982 11381 raft_consensus.cc:686] T d2d8f836c9e547b7a5fa0ba6a1111d0c P eb05f4d58b3f470881b5172e1f312e0d [term 1 LEADER]: Becoming Leader. State: Replica: eb05f4d58b3f470881b5172e1f312e0d, State: Running, Role: LEADER
I20260811 18:55:38.307327 11381 consensus_queue.cc:237] T d2d8f836c9e547b7a5fa0ba6a1111d0c P eb05f4d58b3f470881b5172e1f312e0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7205a2fa560f4b448715b67be73c1174" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 43411 } } peers { permanent_uuid: "1c9d229dc7694099b581b5c31563108d" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 37907 } } peers { permanent_uuid: "eb05f4d58b3f470881b5172e1f312e0d" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38277 } }
I20260811 18:55:38.310452 10460 catalog_manager.cc:5495] T d2d8f836c9e547b7a5fa0ba6a1111d0c P eb05f4d58b3f470881b5172e1f312e0d reported cstate change: term changed from 0 to 1, leader changed from <none> to eb05f4d58b3f470881b5172e1f312e0d (127.27.180.129). New cstate: current_term: 1 leader_uuid: "eb05f4d58b3f470881b5172e1f312e0d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7205a2fa560f4b448715b67be73c1174" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 43411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c9d229dc7694099b581b5c31563108d" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 37907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb05f4d58b3f470881b5172e1f312e0d" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38277 } health_report { overall_health: HEALTHY } } }
W20260811 18:55:38.314843 10831 tablet.cc:2369] T d2d8f836c9e547b7a5fa0ba6a1111d0c P eb05f4d58b3f470881b5172e1f312e0d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:55:38.343538 11370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:38.340178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38600 (local address 127.27.180.129:38277)
0811 18:55:38.340299 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:38.340303 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:38.340316 (+ 13us) server_negotiation.cc:407] Connection header received
0811 18:55:38.340443 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:38.340446 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:38.340501 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:38.340591 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:55:38.340826 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:38.341350 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:38.342188 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:38.342399 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:38.342486 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:38.342760 (+ 274us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:55:38.343168 (+ 408us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:55:38.343268 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:38.343349 (+ 81us) server_negotiation.cc:299] Negotiation successful
0811 18:55:38.343408 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 18:55:38.348997 10996 raft_consensus.cc:1261] T d2d8f836c9e547b7a5fa0ba6a1111d0c P 1c9d229dc7694099b581b5c31563108d [term 1 FOLLOWER]: Refusing update from remote peer eb05f4d58b3f470881b5172e1f312e0d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:55:38.348997 11238 raft_consensus.cc:1261] T d2d8f836c9e547b7a5fa0ba6a1111d0c P 7205a2fa560f4b448715b67be73c1174 [term 1 FOLLOWER]: Refusing update from remote peer eb05f4d58b3f470881b5172e1f312e0d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:55:38.349664 11381 consensus_queue.cc:1035] T d2d8f836c9e547b7a5fa0ba6a1111d0c P eb05f4d58b3f470881b5172e1f312e0d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7205a2fa560f4b448715b67be73c1174" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 43411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:55:38.349841 11393 consensus_queue.cc:1035] T d2d8f836c9e547b7a5fa0ba6a1111d0c P eb05f4d58b3f470881b5172e1f312e0d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c9d229dc7694099b581b5c31563108d" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 37907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:55:38.356973 11398 mvcc.cc:205] Tried to move back new op lower bound from 7317399709070712832 to 7317399708909461504. Current Snapshot: MvccSnapshot[applied={T|T < 7317399709070712832}]
I20260811 18:55:38.357429 11399 mvcc.cc:205] Tried to move back new op lower bound from 7317399709070712832 to 7317399708909461504. Current Snapshot: MvccSnapshot[applied={T|T < 7317399709070712832}]
I20260811 18:55:38.366283 10462 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35620:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:55:38.366420 10462 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35620:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:55:38.368654 10462 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8627c387211147ed82345656961a4525: Sending DeleteTablet for 3 replicas of tablet d2d8f836c9e547b7a5fa0ba6a1111d0c
I20260811 18:55:38.369423 10958 tablet_service.cc:1514] Processing DeleteTablet for tablet d2d8f836c9e547b7a5fa0ba6a1111d0c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:55:38 UTC) from {username='oryx', principal='oryx/127.27.180.190@KRBTEST.COM'} at 127.0.0.1:57478
I20260811 18:55:38.369429 10721 tablet_service.cc:1514] Processing DeleteTablet for tablet d2d8f836c9e547b7a5fa0ba6a1111d0c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:55:38 UTC) from {username='oryx', principal='oryx/127.27.180.190@KRBTEST.COM'} at 127.0.0.1:38594
I20260811 18:55:38.369920 11422 tablet_replica.cc:330] T d2d8f836c9e547b7a5fa0ba6a1111d0c P 1c9d229dc7694099b581b5c31563108d: stopping tablet replica
I20260811 18:55:38.369912 11209 tablet_service.cc:1514] Processing DeleteTablet for tablet d2d8f836c9e547b7a5fa0ba6a1111d0c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:55:38 UTC) from {username='oryx', principal='oryx/127.27.180.190@KRBTEST.COM'} at 127.0.0.1:37328
I20260811 18:55:38.370199 11421 tablet_replica.cc:330] T d2d8f836c9e547b7a5fa0ba6a1111d0c P eb05f4d58b3f470881b5172e1f312e0d: stopping tablet replica
I20260811 18:55:38.370465 11424 tablet_replica.cc:330] T d2d8f836c9e547b7a5fa0ba6a1111d0c P 7205a2fa560f4b448715b67be73c1174: stopping tablet replica
I20260811 18:55:38.370509 11421 raft_consensus.cc:2229] T d2d8f836c9e547b7a5fa0ba6a1111d0c P eb05f4d58b3f470881b5172e1f312e0d [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:55:38.370724 11424 raft_consensus.cc:2229] T d2d8f836c9e547b7a5fa0ba6a1111d0c P 7205a2fa560f4b448715b67be73c1174 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:55:38.370726 10462 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35620:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 18:55:38.370867 11421 raft_consensus.cc:2258] T d2d8f836c9e547b7a5fa0ba6a1111d0c P eb05f4d58b3f470881b5172e1f312e0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:55:38.371011 11424 raft_consensus.cc:2258] T d2d8f836c9e547b7a5fa0ba6a1111d0c P 7205a2fa560f4b448715b67be73c1174 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260811 18:55:38.371090 10462 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:55:38.371282 11422 raft_consensus.cc:2229] T d2d8f836c9e547b7a5fa0ba6a1111d0c P 1c9d229dc7694099b581b5c31563108d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:55:38.371559 11422 raft_consensus.cc:2258] T d2d8f836c9e547b7a5fa0ba6a1111d0c P 1c9d229dc7694099b581b5c31563108d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:55:38.373047 11421 ts_tablet_manager.cc:1905] T d2d8f836c9e547b7a5fa0ba6a1111d0c P eb05f4d58b3f470881b5172e1f312e0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:55:38.373261 11422 ts_tablet_manager.cc:1905] T d2d8f836c9e547b7a5fa0ba6a1111d0c P 1c9d229dc7694099b581b5c31563108d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:55:38.375835 11421 ts_tablet_manager.cc:1918] T d2d8f836c9e547b7a5fa0ba6a1111d0c P eb05f4d58b3f470881b5172e1f312e0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:55:38.375933 11421 log.cc:1196] T d2d8f836c9e547b7a5fa0ba6a1111d0c P eb05f4d58b3f470881b5172e1f312e0d: Deleting WAL directory at /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/minicluster-data/ts-0/wal/wals/d2d8f836c9e547b7a5fa0ba6a1111d0c
I20260811 18:55:38.376098 10958 tablet_service.cc:1467] Processing CreateTablet for tablet adee31878d5647fa90e59d3a6c78cb95 (DEFAULT_TABLE table=test-table [id=ba3fbc7c760e41ad80cc7c9d98a29499]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:55:38.376215 11421 ts_tablet_manager.cc:1939] T d2d8f836c9e547b7a5fa0ba6a1111d0c P eb05f4d58b3f470881b5172e1f312e0d: Deleting consensus metadata
I20260811 18:55:38.376355 10958 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet adee31878d5647fa90e59d3a6c78cb95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:55:38.376791 11424 ts_tablet_manager.cc:1905] T d2d8f836c9e547b7a5fa0ba6a1111d0c P 7205a2fa560f4b448715b67be73c1174: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:55:38.378443 11372 tablet_bootstrap.cc:492] T adee31878d5647fa90e59d3a6c78cb95 P 1c9d229dc7694099b581b5c31563108d: Bootstrap starting.
I20260811 18:55:38.378609 10721 tablet_service.cc:1467] Processing CreateTablet for tablet adee31878d5647fa90e59d3a6c78cb95 (DEFAULT_TABLE table=test-table [id=ba3fbc7c760e41ad80cc7c9d98a29499]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:55:38.378826 10721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet adee31878d5647fa90e59d3a6c78cb95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:55:38.379307 11424 ts_tablet_manager.cc:1918] T d2d8f836c9e547b7a5fa0ba6a1111d0c P 7205a2fa560f4b448715b67be73c1174: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:55:38.379343 11372 tablet_bootstrap.cc:654] T adee31878d5647fa90e59d3a6c78cb95 P 1c9d229dc7694099b581b5c31563108d: Neither blocks nor log segments found. Creating new log.
I20260811 18:55:38.379388 11424 log.cc:1196] T d2d8f836c9e547b7a5fa0ba6a1111d0c P 7205a2fa560f4b448715b67be73c1174: Deleting WAL directory at /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/minicluster-data/ts-2/wal/wals/d2d8f836c9e547b7a5fa0ba6a1111d0c
I20260811 18:55:38.379673 11424 ts_tablet_manager.cc:1939] T d2d8f836c9e547b7a5fa0ba6a1111d0c P 7205a2fa560f4b448715b67be73c1174: Deleting consensus metadata
I20260811 18:55:38.380841 10430 catalog_manager.cc:4841] TS 7205a2fa560f4b448715b67be73c1174 (127.27.180.131:43411): tablet d2d8f836c9e547b7a5fa0ba6a1111d0c (table test-table [id=b398c77b16b74162ac8d38f6399bc082]) successfully deleted
I20260811 18:55:38.380940 11373 tablet_bootstrap.cc:492] T adee31878d5647fa90e59d3a6c78cb95 P eb05f4d58b3f470881b5172e1f312e0d: Bootstrap starting.
I20260811 18:55:38.381191 11209 tablet_service.cc:1467] Processing CreateTablet for tablet adee31878d5647fa90e59d3a6c78cb95 (DEFAULT_TABLE table=test-table [id=ba3fbc7c760e41ad80cc7c9d98a29499]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:55:38.381394 11209 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet adee31878d5647fa90e59d3a6c78cb95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:55:38.381932 11373 tablet_bootstrap.cc:654] T adee31878d5647fa90e59d3a6c78cb95 P eb05f4d58b3f470881b5172e1f312e0d: Neither blocks nor log segments found. Creating new log.
I20260811 18:55:38.383061 11371 tablet_bootstrap.cc:492] T adee31878d5647fa90e59d3a6c78cb95 P 7205a2fa560f4b448715b67be73c1174: Bootstrap starting.
I20260811 18:55:38.383082 11373 tablet_bootstrap.cc:492] T adee31878d5647fa90e59d3a6c78cb95 P eb05f4d58b3f470881b5172e1f312e0d: No bootstrap required, opened a new log
I20260811 18:55:38.383152 11373 ts_tablet_manager.cc:1397] T adee31878d5647fa90e59d3a6c78cb95 P eb05f4d58b3f470881b5172e1f312e0d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:55:38.383577 11373 raft_consensus.cc:348] T adee31878d5647fa90e59d3a6c78cb95 P eb05f4d58b3f470881b5172e1f312e0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb05f4d58b3f470881b5172e1f312e0d" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38277 } } peers { permanent_uuid: "1c9d229dc7694099b581b5c31563108d" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 37907 } } peers { permanent_uuid: "7205a2fa560f4b448715b67be73c1174" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 43411 } }
I20260811 18:55:38.383685 11373 raft_consensus.cc:374] T adee31878d5647fa90e59d3a6c78cb95 P eb05f4d58b3f470881b5172e1f312e0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:55:38.383714 11373 raft_consensus.cc:729] T adee31878d5647fa90e59d3a6c78cb95 P eb05f4d58b3f470881b5172e1f312e0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb05f4d58b3f470881b5172e1f312e0d, State: Initialized, Role: FOLLOWER
I20260811 18:55:38.383819 11373 consensus_queue.cc:260] T adee31878d5647fa90e59d3a6c78cb95 P eb05f4d58b3f470881b5172e1f312e0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb05f4d58b3f470881b5172e1f312e0d" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38277 } } peers { permanent_uuid: "1c9d229dc7694099b581b5c31563108d" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 37907 } } peers { permanent_uuid: "7205a2fa560f4b448715b67be73c1174" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 43411 } }
I20260811 18:55:38.383956 11371 tablet_bootstrap.cc:654] T adee31878d5647fa90e59d3a6c78cb95 P 7205a2fa560f4b448715b67be73c1174: Neither blocks nor log segments found. Creating new log.
I20260811 18:55:38.383935 10430 catalog_manager.cc:4841] TS eb05f4d58b3f470881b5172e1f312e0d (127.27.180.129:38277): tablet d2d8f836c9e547b7a5fa0ba6a1111d0c (table test-table [id=b398c77b16b74162ac8d38f6399bc082]) successfully deleted
I20260811 18:55:38.384054 11373 ts_tablet_manager.cc:1428] T adee31878d5647fa90e59d3a6c78cb95 P eb05f4d58b3f470881b5172e1f312e0d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 18:55:38.385016 11371 tablet_bootstrap.cc:492] T adee31878d5647fa90e59d3a6c78cb95 P 7205a2fa560f4b448715b67be73c1174: No bootstrap required, opened a new log
I20260811 18:55:38.385126 11371 ts_tablet_manager.cc:1397] T adee31878d5647fa90e59d3a6c78cb95 P 7205a2fa560f4b448715b67be73c1174: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:55:38.385543 11371 raft_consensus.cc:348] T adee31878d5647fa90e59d3a6c78cb95 P 7205a2fa560f4b448715b67be73c1174 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb05f4d58b3f470881b5172e1f312e0d" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38277 } } peers { permanent_uuid: "1c9d229dc7694099b581b5c31563108d" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 37907 } } peers { permanent_uuid: "7205a2fa560f4b448715b67be73c1174" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 43411 } }
I20260811 18:55:38.385638 11371 raft_consensus.cc:374] T adee31878d5647fa90e59d3a6c78cb95 P 7205a2fa560f4b448715b67be73c1174 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:55:38.385670 11371 raft_consensus.cc:729] T adee31878d5647fa90e59d3a6c78cb95 P 7205a2fa560f4b448715b67be73c1174 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7205a2fa560f4b448715b67be73c1174, State: Initialized, Role: FOLLOWER
I20260811 18:55:38.385788 11371 consensus_queue.cc:260] T adee31878d5647fa90e59d3a6c78cb95 P 7205a2fa560f4b448715b67be73c1174 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb05f4d58b3f470881b5172e1f312e0d" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38277 } } peers { permanent_uuid: "1c9d229dc7694099b581b5c31563108d" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 37907 } } peers { permanent_uuid: "7205a2fa560f4b448715b67be73c1174" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 43411 } }
I20260811 18:55:38.386062 11371 ts_tablet_manager.cc:1428] T adee31878d5647fa90e59d3a6c78cb95 P 7205a2fa560f4b448715b67be73c1174: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 18:55:38.388312 11422 ts_tablet_manager.cc:1918] T d2d8f836c9e547b7a5fa0ba6a1111d0c P 1c9d229dc7694099b581b5c31563108d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:55:38.388401 11422 log.cc:1196] T d2d8f836c9e547b7a5fa0ba6a1111d0c P 1c9d229dc7694099b581b5c31563108d: Deleting WAL directory at /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/minicluster-data/ts-1/wal/wals/d2d8f836c9e547b7a5fa0ba6a1111d0c
I20260811 18:55:38.388952 11422 ts_tablet_manager.cc:1939] T d2d8f836c9e547b7a5fa0ba6a1111d0c P 1c9d229dc7694099b581b5c31563108d: Deleting consensus metadata
I20260811 18:55:38.389982 10436 catalog_manager.cc:4841] TS 1c9d229dc7694099b581b5c31563108d (127.27.180.130:37907): tablet d2d8f836c9e547b7a5fa0ba6a1111d0c (table test-table [id=b398c77b16b74162ac8d38f6399bc082]) successfully deleted
I20260811 18:55:38.390564 11372 tablet_bootstrap.cc:492] T adee31878d5647fa90e59d3a6c78cb95 P 1c9d229dc7694099b581b5c31563108d: No bootstrap required, opened a new log
I20260811 18:55:38.390654 11372 ts_tablet_manager.cc:1397] T adee31878d5647fa90e59d3a6c78cb95 P 1c9d229dc7694099b581b5c31563108d: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.000s
I20260811 18:55:38.391068 11372 raft_consensus.cc:348] T adee31878d5647fa90e59d3a6c78cb95 P 1c9d229dc7694099b581b5c31563108d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb05f4d58b3f470881b5172e1f312e0d" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38277 } } peers { permanent_uuid: "1c9d229dc7694099b581b5c31563108d" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 37907 } } peers { permanent_uuid: "7205a2fa560f4b448715b67be73c1174" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 43411 } }
I20260811 18:55:38.391179 11372 raft_consensus.cc:374] T adee31878d5647fa90e59d3a6c78cb95 P 1c9d229dc7694099b581b5c31563108d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:55:38.391217 11372 raft_consensus.cc:729] T adee31878d5647fa90e59d3a6c78cb95 P 1c9d229dc7694099b581b5c31563108d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c9d229dc7694099b581b5c31563108d, State: Initialized, Role: FOLLOWER
I20260811 18:55:38.391331 11372 consensus_queue.cc:260] T adee31878d5647fa90e59d3a6c78cb95 P 1c9d229dc7694099b581b5c31563108d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb05f4d58b3f470881b5172e1f312e0d" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38277 } } peers { permanent_uuid: "1c9d229dc7694099b581b5c31563108d" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 37907 } } peers { permanent_uuid: "7205a2fa560f4b448715b67be73c1174" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 43411 } }
I20260811 18:55:38.391588 11372 ts_tablet_manager.cc:1428] T adee31878d5647fa90e59d3a6c78cb95 P 1c9d229dc7694099b581b5c31563108d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 18:55:38.589581 11386 raft_consensus.cc:482] T adee31878d5647fa90e59d3a6c78cb95 P 7205a2fa560f4b448715b67be73c1174 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:55:38.589735 11386 raft_consensus.cc:504] T adee31878d5647fa90e59d3a6c78cb95 P 7205a2fa560f4b448715b67be73c1174 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb05f4d58b3f470881b5172e1f312e0d" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38277 } } peers { permanent_uuid: "1c9d229dc7694099b581b5c31563108d" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 37907 } } peers { permanent_uuid: "7205a2fa560f4b448715b67be73c1174" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 43411 } }
I20260811 18:55:38.590750 11386 leader_election.cc:288] T adee31878d5647fa90e59d3a6c78cb95 P 7205a2fa560f4b448715b67be73c1174 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb05f4d58b3f470881b5172e1f312e0d (127.27.180.129:38277), 1c9d229dc7694099b581b5c31563108d (127.27.180.130:37907)
I20260811 18:55:38.594717 11370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:38.591091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.131:39843 (local address 127.27.180.129:38277)
0811 18:55:38.591224 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:38.591229 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:38.591376 (+ 147us) server_negotiation.cc:407] Connection header received
0811 18:55:38.591540 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:38.591542 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:38.591586 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:38.591650 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:55:38.591918 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:38.592411 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:38.593585 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:38.594191 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:38.594291 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:38.594335 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:38.594526 (+ 191us) server_negotiation.cc:299] Negotiation successful
0811 18:55:38.594594 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 18:55:38.594717 11368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:38.591091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.131:40143 (local address 127.27.180.130:37907)
0811 18:55:38.591343 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:38.591347 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:38.591465 (+ 118us) server_negotiation.cc:407] Connection header received
0811 18:55:38.591593 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:38.591596 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:38.591638 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:38.591699 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:55:38.591939 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:38.592419 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:38.593542 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:38.594154 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:38.594275 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:38.594329 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:38.594526 (+ 197us) server_negotiation.cc:299] Negotiation successful
0811 18:55:38.594592 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138}
I20260811 18:55:38.594777 11429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:38.590998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.130:37907 (local address 127.27.180.131:40143)
0811 18:55:38.591375 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:38.591396 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:38.591495 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:38.591716 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:38.591718 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:38.591733 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:55:38.591837 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:38.591842 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:38.592513 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:38.592517 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:38.593411 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:38.593420 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:38.594269 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:38.594273 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:38.594386 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:38.594415 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 18:55:38.594490 (+ 75us) client_negotiation.cc:240] Negotiation successful
0811 18:55:38.594627 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"mutex_wait_us":88,"thread_start_us":65,"threads_started":1}
I20260811 18:55:38.594933 11428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:38.590999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.129:38277 (local address 127.27.180.131:39843)
0811 18:55:38.591285 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:38.591299 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:38.591434 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:38.591673 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:38.591676 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:38.591691 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:55:38.591806 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:38.591812 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:38.592514 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:38.592517 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:38.593467 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:38.593477 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:38.594274 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:38.594277 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:38.594391 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:38.594425 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 18:55:38.594490 (+ 65us) client_negotiation.cc:240] Negotiation successful
0811 18:55:38.594595 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215,"mutex_wait_us":82,"thread_start_us":72,"threads_started":1}
I20260811 18:55:38.595294 10995 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "adee31878d5647fa90e59d3a6c78cb95" candidate_uuid: "7205a2fa560f4b448715b67be73c1174" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c9d229dc7694099b581b5c31563108d" is_pre_election: true
I20260811 18:55:38.595453 10995 raft_consensus.cc:2454] T adee31878d5647fa90e59d3a6c78cb95 P 1c9d229dc7694099b581b5c31563108d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7205a2fa560f4b448715b67be73c1174 in term 0.
I20260811 18:55:38.595587 10758 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "adee31878d5647fa90e59d3a6c78cb95" candidate_uuid: "7205a2fa560f4b448715b67be73c1174" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb05f4d58b3f470881b5172e1f312e0d" is_pre_election: true
I20260811 18:55:38.595817 10758 raft_consensus.cc:2454] T adee31878d5647fa90e59d3a6c78cb95 P eb05f4d58b3f470881b5172e1f312e0d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7205a2fa560f4b448715b67be73c1174 in term 0.
I20260811 18:55:38.595817 11122 leader_election.cc:302] T adee31878d5647fa90e59d3a6c78cb95 P 7205a2fa560f4b448715b67be73c1174 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c9d229dc7694099b581b5c31563108d, 7205a2fa560f4b448715b67be73c1174; no voters:
I20260811 18:55:38.596087 11386 raft_consensus.cc:2781] T adee31878d5647fa90e59d3a6c78cb95 P 7205a2fa560f4b448715b67be73c1174 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:55:38.596149 11386 raft_consensus.cc:482] T adee31878d5647fa90e59d3a6c78cb95 P 7205a2fa560f4b448715b67be73c1174 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:55:38.596187 11386 raft_consensus.cc:3037] T adee31878d5647fa90e59d3a6c78cb95 P 7205a2fa560f4b448715b67be73c1174 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:55:38.597222 11386 raft_consensus.cc:504] T adee31878d5647fa90e59d3a6c78cb95 P 7205a2fa560f4b448715b67be73c1174 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb05f4d58b3f470881b5172e1f312e0d" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38277 } } peers { permanent_uuid: "1c9d229dc7694099b581b5c31563108d" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 37907 } } peers { permanent_uuid: "7205a2fa560f4b448715b67be73c1174" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 43411 } }
I20260811 18:55:38.597543 11386 leader_election.cc:288] T adee31878d5647fa90e59d3a6c78cb95 P 7205a2fa560f4b448715b67be73c1174 [CANDIDATE]: Term 1 election: Requested vote from peers eb05f4d58b3f470881b5172e1f312e0d (127.27.180.129:38277), 1c9d229dc7694099b581b5c31563108d (127.27.180.130:37907)
I20260811 18:55:38.597859 10995 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "adee31878d5647fa90e59d3a6c78cb95" candidate_uuid: "7205a2fa560f4b448715b67be73c1174" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c9d229dc7694099b581b5c31563108d"
I20260811 18:55:38.597859 10758 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "adee31878d5647fa90e59d3a6c78cb95" candidate_uuid: "7205a2fa560f4b448715b67be73c1174" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb05f4d58b3f470881b5172e1f312e0d"
I20260811 18:55:38.597982 10995 raft_consensus.cc:3037] T adee31878d5647fa90e59d3a6c78cb95 P 1c9d229dc7694099b581b5c31563108d [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:55:38.597982 10758 raft_consensus.cc:3037] T adee31878d5647fa90e59d3a6c78cb95 P eb05f4d58b3f470881b5172e1f312e0d [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:55:38.599090 10995 raft_consensus.cc:2454] T adee31878d5647fa90e59d3a6c78cb95 P 1c9d229dc7694099b581b5c31563108d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7205a2fa560f4b448715b67be73c1174 in term 1.
I20260811 18:55:38.599090 10758 raft_consensus.cc:2454] T adee31878d5647fa90e59d3a6c78cb95 P eb05f4d58b3f470881b5172e1f312e0d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7205a2fa560f4b448715b67be73c1174 in term 1.
I20260811 18:55:38.599357 11122 leader_election.cc:302] T adee31878d5647fa90e59d3a6c78cb95 P 7205a2fa560f4b448715b67be73c1174 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1c9d229dc7694099b581b5c31563108d, 7205a2fa560f4b448715b67be73c1174, eb05f4d58b3f470881b5172e1f312e0d; no voters:
I20260811 18:55:38.599515 11386 raft_consensus.cc:2781] T adee31878d5647fa90e59d3a6c78cb95 P 7205a2fa560f4b448715b67be73c1174 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:55:38.599670 11386 raft_consensus.cc:686] T adee31878d5647fa90e59d3a6c78cb95 P 7205a2fa560f4b448715b67be73c1174 [term 1 LEADER]: Becoming Leader. State: Replica: 7205a2fa560f4b448715b67be73c1174, State: Running, Role: LEADER
I20260811 18:55:38.599888 11386 consensus_queue.cc:237] T adee31878d5647fa90e59d3a6c78cb95 P 7205a2fa560f4b448715b67be73c1174 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb05f4d58b3f470881b5172e1f312e0d" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38277 } } peers { permanent_uuid: "1c9d229dc7694099b581b5c31563108d" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 37907 } } peers { permanent_uuid: "7205a2fa560f4b448715b67be73c1174" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 43411 } }
I20260811 18:55:38.602108 10460 catalog_manager.cc:5495] T adee31878d5647fa90e59d3a6c78cb95 P 7205a2fa560f4b448715b67be73c1174 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7205a2fa560f4b448715b67be73c1174 (127.27.180.131). New cstate: current_term: 1 leader_uuid: "7205a2fa560f4b448715b67be73c1174" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb05f4d58b3f470881b5172e1f312e0d" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c9d229dc7694099b581b5c31563108d" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 37907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7205a2fa560f4b448715b67be73c1174" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 43411 } health_report { overall_health: HEALTHY } } }
Aug 11 18:55:38 dist-test-slave-2j7r krb5kdc[10385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474537, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.180.190@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM
I20260811 18:55:38.646088 11443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:38.638668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:37195 (local address 127.0.0.1:35628)
0811 18:55:38.638962 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:38.638975 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:38.639069 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:38.639273 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:38.639274 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:38.639656 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:38.639748 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:38.639754 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:38.640436 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:38.640439 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:38.640985 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:38.640991 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:38.641401 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:38.641404 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:38.641460 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:38.641935 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:38.641946 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:38.643261 (+ 1315us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:55:38.644743 (+ 1482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:38.644749 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:38.644754 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:38.644972 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:38.645211 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:38.645213 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:38.645215 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:38.645298 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:38.645605 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:55:38.645610 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:55:38.645708 (+ 98us) client_negotiation.cc:769] Sending connection context
0811 18:55:38.645796 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 18:55:38.645933 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":98,"threads_started":1}
I20260811 18:55:38.646183 11450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:38.638732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35628 (local address 127.27.180.190:37195)
0811 18:55:38.638960 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:38.638969 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:38.639040 (+ 71us) server_negotiation.cc:407] Connection header received
0811 18:55:38.639132 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:38.639134 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:38.639173 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:38.639250 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:38.639835 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:38.640361 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:38.641060 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:38.641318 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:38.641390 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:38.643379 (+ 1989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:38.643382 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:38.643384 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:38.643406 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:38.644662 (+ 1256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:38.645041 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:38.645044 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:38.645045 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:38.645140 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:38.645365 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:38.645367 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:38.645368 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:38.645528 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:38.645600 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:38.645835 (+ 235us) server_negotiation.cc:299] Negotiation successful
0811 18:55:38.645990 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":55,"server-negotiator.queue_time_us":159,"thread_start_us":79,"threads_started":1}
I20260811 18:55:38.650036 10460 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.27.180.190@KRBTEST.COM'} at 127.0.0.1:35628:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 18:55:38.650784 10460 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:55:38.655578 10721 tablet_service.cc:1467] Processing CreateTablet for tablet 668822252ffb4af7a2548fda9d4f7b6e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=06395d107c0448db859bce33105b4b53]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 18:55:38.655622 10958 tablet_service.cc:1467] Processing CreateTablet for tablet 668822252ffb4af7a2548fda9d4f7b6e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=06395d107c0448db859bce33105b4b53]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 18:55:38.656018 10721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 668822252ffb4af7a2548fda9d4f7b6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:55:38.656020 10958 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 668822252ffb4af7a2548fda9d4f7b6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:55:38.656001 11209 tablet_service.cc:1467] Processing CreateTablet for tablet 668822252ffb4af7a2548fda9d4f7b6e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=06395d107c0448db859bce33105b4b53]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 18:55:38.656366 11209 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 668822252ffb4af7a2548fda9d4f7b6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:55:38.658586 11372 tablet_bootstrap.cc:492] T 668822252ffb4af7a2548fda9d4f7b6e P 1c9d229dc7694099b581b5c31563108d: Bootstrap starting.
I20260811 18:55:38.659114 11371 tablet_bootstrap.cc:492] T 668822252ffb4af7a2548fda9d4f7b6e P 7205a2fa560f4b448715b67be73c1174: Bootstrap starting.
I20260811 18:55:38.659600 11372 tablet_bootstrap.cc:654] T 668822252ffb4af7a2548fda9d4f7b6e P 1c9d229dc7694099b581b5c31563108d: Neither blocks nor log segments found. Creating new log.
I20260811 18:55:38.660043 11371 tablet_bootstrap.cc:654] T 668822252ffb4af7a2548fda9d4f7b6e P 7205a2fa560f4b448715b67be73c1174: Neither blocks nor log segments found. Creating new log.
I20260811 18:55:38.660250 11373 tablet_bootstrap.cc:492] T 668822252ffb4af7a2548fda9d4f7b6e P eb05f4d58b3f470881b5172e1f312e0d: Bootstrap starting.
I20260811 18:55:38.660713 11372 tablet_bootstrap.cc:492] T 668822252ffb4af7a2548fda9d4f7b6e P 1c9d229dc7694099b581b5c31563108d: No bootstrap required, opened a new log
I20260811 18:55:38.660772 11372 ts_tablet_manager.cc:1397] T 668822252ffb4af7a2548fda9d4f7b6e P 1c9d229dc7694099b581b5c31563108d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:55:38.660982 11373 tablet_bootstrap.cc:654] T 668822252ffb4af7a2548fda9d4f7b6e P eb05f4d58b3f470881b5172e1f312e0d: Neither blocks nor log segments found. Creating new log.
I20260811 18:55:38.661175 11372 raft_consensus.cc:348] T 668822252ffb4af7a2548fda9d4f7b6e P 1c9d229dc7694099b581b5c31563108d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb05f4d58b3f470881b5172e1f312e0d" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38277 } } peers { permanent_uuid: "1c9d229dc7694099b581b5c31563108d" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 37907 } } peers { permanent_uuid: "7205a2fa560f4b448715b67be73c1174" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 43411 } }
I20260811 18:55:38.661281 11372 raft_consensus.cc:374] T 668822252ffb4af7a2548fda9d4f7b6e P 1c9d229dc7694099b581b5c31563108d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:55:38.661314 11372 raft_consensus.cc:729] T 668822252ffb4af7a2548fda9d4f7b6e P 1c9d229dc7694099b581b5c31563108d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c9d229dc7694099b581b5c31563108d, State: Initialized, Role: FOLLOWER
I20260811 18:55:38.661438 11372 consensus_queue.cc:260] T 668822252ffb4af7a2548fda9d4f7b6e P 1c9d229dc7694099b581b5c31563108d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb05f4d58b3f470881b5172e1f312e0d" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38277 } } peers { permanent_uuid: "1c9d229dc7694099b581b5c31563108d" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 37907 } } peers { permanent_uuid: "7205a2fa560f4b448715b67be73c1174" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 43411 } }
I20260811 18:55:38.661715 11372 ts_tablet_manager.cc:1428] T 668822252ffb4af7a2548fda9d4f7b6e P 1c9d229dc7694099b581b5c31563108d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 18:55:38.661702 11387 tablet_replica.cc:439] T 668822252ffb4af7a2548fda9d4f7b6e P 1c9d229dc7694099b581b5c31563108d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb05f4d58b3f470881b5172e1f312e0d" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38277 } } peers { permanent_uuid: "1c9d229dc7694099b581b5c31563108d" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 37907 } } peers { permanent_uuid: "7205a2fa560f4b448715b67be73c1174" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 43411 } } }
I20260811 18:55:38.661861 11387 tablet_replica.cc:442] T 668822252ffb4af7a2548fda9d4f7b6e P 1c9d229dc7694099b581b5c31563108d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 18:55:38.662125 11373 tablet_bootstrap.cc:492] T 668822252ffb4af7a2548fda9d4f7b6e P eb05f4d58b3f470881b5172e1f312e0d: No bootstrap required, opened a new log
I20260811 18:55:38.662199 11373 ts_tablet_manager.cc:1397] T 668822252ffb4af7a2548fda9d4f7b6e P eb05f4d58b3f470881b5172e1f312e0d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 18:55:38.662592 11373 raft_consensus.cc:348] T 668822252ffb4af7a2548fda9d4f7b6e P eb05f4d58b3f470881b5172e1f312e0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb05f4d58b3f470881b5172e1f312e0d" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38277 } } peers { permanent_uuid: "1c9d229dc7694099b581b5c31563108d" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 37907 } } peers { permanent_uuid: "7205a2fa560f4b448715b67be73c1174" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 43411 } }
I20260811 18:55:38.662711 11373 raft_consensus.cc:374] T 668822252ffb4af7a2548fda9d4f7b6e P eb05f4d58b3f470881b5172e1f312e0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:55:38.662748 11373 raft_consensus.cc:729] T 668822252ffb4af7a2548fda9d4f7b6e P eb05f4d58b3f470881b5172e1f312e0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb05f4d58b3f470881b5172e1f312e0d, State: Initialized, Role: FOLLOWER
I20260811 18:55:38.662874 11373 consensus_queue.cc:260] T 668822252ffb4af7a2548fda9d4f7b6e P eb05f4d58b3f470881b5172e1f312e0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb05f4d58b3f470881b5172e1f312e0d" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38277 } } peers { permanent_uuid: "1c9d229dc7694099b581b5c31563108d" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 37907 } } peers { permanent_uuid: "7205a2fa560f4b448715b67be73c1174" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 43411 } }
I20260811 18:55:38.663081 11371 tablet_bootstrap.cc:492] T 668822252ffb4af7a2548fda9d4f7b6e P 7205a2fa560f4b448715b67be73c1174: No bootstrap required, opened a new log
I20260811 18:55:38.663131 11373 ts_tablet_manager.cc:1428] T 668822252ffb4af7a2548fda9d4f7b6e P eb05f4d58b3f470881b5172e1f312e0d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 18:55:38.663090 11393 tablet_replica.cc:439] T 668822252ffb4af7a2548fda9d4f7b6e P eb05f4d58b3f470881b5172e1f312e0d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb05f4d58b3f470881b5172e1f312e0d" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38277 } } peers { permanent_uuid: "1c9d229dc7694099b581b5c31563108d" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 37907 } } peers { permanent_uuid: "7205a2fa560f4b448715b67be73c1174" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 43411 } } }
I20260811 18:55:38.663173 11371 ts_tablet_manager.cc:1397] T 668822252ffb4af7a2548fda9d4f7b6e P 7205a2fa560f4b448715b67be73c1174: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260811 18:55:38.663205 11393 tablet_replica.cc:442] T 668822252ffb4af7a2548fda9d4f7b6e P eb05f4d58b3f470881b5172e1f312e0d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 18:55:38.663573 11371 raft_consensus.cc:348] T 668822252ffb4af7a2548fda9d4f7b6e P 7205a2fa560f4b448715b67be73c1174 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb05f4d58b3f470881b5172e1f312e0d" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38277 } } peers { permanent_uuid: "1c9d229dc7694099b581b5c31563108d" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 37907 } } peers { permanent_uuid: "7205a2fa560f4b448715b67be73c1174" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 43411 } }
I20260811 18:55:38.663676 11371 raft_consensus.cc:374] T 668822252ffb4af7a2548fda9d4f7b6e P 7205a2fa560f4b448715b67be73c1174 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:55:38.663707 11371 raft_consensus.cc:729] T 668822252ffb4af7a2548fda9d4f7b6e P 7205a2fa560f4b448715b67be73c1174 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7205a2fa560f4b448715b67be73c1174, State: Initialized, Role: FOLLOWER
I20260811 18:55:38.663841 11371 consensus_queue.cc:260] T 668822252ffb4af7a2548fda9d4f7b6e P 7205a2fa560f4b448715b67be73c1174 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb05f4d58b3f470881b5172e1f312e0d" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38277 } } peers { permanent_uuid: "1c9d229dc7694099b581b5c31563108d" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 37907 } } peers { permanent_uuid: "7205a2fa560f4b448715b67be73c1174" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 43411 } }
I20260811 18:55:38.664098 11371 ts_tablet_manager.cc:1428] T 668822252ffb4af7a2548fda9d4f7b6e P 7205a2fa560f4b448715b67be73c1174: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 18:55:38.664100 11386 tablet_replica.cc:439] T 668822252ffb4af7a2548fda9d4f7b6e P 7205a2fa560f4b448715b67be73c1174: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb05f4d58b3f470881b5172e1f312e0d" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38277 } } peers { permanent_uuid: "1c9d229dc7694099b581b5c31563108d" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 37907 } } peers { permanent_uuid: "7205a2fa560f4b448715b67be73c1174" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 43411 } } }
I20260811 18:55:38.664259 11386 tablet_replica.cc:442] T 668822252ffb4af7a2548fda9d4f7b6e P 7205a2fa560f4b448715b67be73c1174: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 18:55:38.696310 11386 raft_consensus.cc:482] T 668822252ffb4af7a2548fda9d4f7b6e P 7205a2fa560f4b448715b67be73c1174 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:55:38.696455 11386 raft_consensus.cc:504] T 668822252ffb4af7a2548fda9d4f7b6e P 7205a2fa560f4b448715b67be73c1174 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb05f4d58b3f470881b5172e1f312e0d" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38277 } } peers { permanent_uuid: "1c9d229dc7694099b581b5c31563108d" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 37907 } } peers { permanent_uuid: "7205a2fa560f4b448715b67be73c1174" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 43411 } }
I20260811 18:55:38.696844 11386 leader_election.cc:288] T 668822252ffb4af7a2548fda9d4f7b6e P 7205a2fa560f4b448715b67be73c1174 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb05f4d58b3f470881b5172e1f312e0d (127.27.180.129:38277), 1c9d229dc7694099b581b5c31563108d (127.27.180.130:37907)
I20260811 18:55:38.697317 10758 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "668822252ffb4af7a2548fda9d4f7b6e" candidate_uuid: "7205a2fa560f4b448715b67be73c1174" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb05f4d58b3f470881b5172e1f312e0d" is_pre_election: true
I20260811 18:55:38.697341 10995 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "668822252ffb4af7a2548fda9d4f7b6e" candidate_uuid: "7205a2fa560f4b448715b67be73c1174" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c9d229dc7694099b581b5c31563108d" is_pre_election: true
I20260811 18:55:38.697469 10758 raft_consensus.cc:2454] T 668822252ffb4af7a2548fda9d4f7b6e P eb05f4d58b3f470881b5172e1f312e0d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7205a2fa560f4b448715b67be73c1174 in term 0.
I20260811 18:55:38.697487 10995 raft_consensus.cc:2454] T 668822252ffb4af7a2548fda9d4f7b6e P 1c9d229dc7694099b581b5c31563108d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7205a2fa560f4b448715b67be73c1174 in term 0.
I20260811 18:55:38.697767 11122 leader_election.cc:302] T 668822252ffb4af7a2548fda9d4f7b6e P 7205a2fa560f4b448715b67be73c1174 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1c9d229dc7694099b581b5c31563108d, 7205a2fa560f4b448715b67be73c1174, eb05f4d58b3f470881b5172e1f312e0d; no voters:
I20260811 18:55:38.697943 11386 raft_consensus.cc:2781] T 668822252ffb4af7a2548fda9d4f7b6e P 7205a2fa560f4b448715b67be73c1174 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:55:38.698004 11386 raft_consensus.cc:482] T 668822252ffb4af7a2548fda9d4f7b6e P 7205a2fa560f4b448715b67be73c1174 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:55:38.698050 11386 raft_consensus.cc:3037] T 668822252ffb4af7a2548fda9d4f7b6e P 7205a2fa560f4b448715b67be73c1174 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:55:38.699070 11386 raft_consensus.cc:504] T 668822252ffb4af7a2548fda9d4f7b6e P 7205a2fa560f4b448715b67be73c1174 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb05f4d58b3f470881b5172e1f312e0d" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38277 } } peers { permanent_uuid: "1c9d229dc7694099b581b5c31563108d" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 37907 } } peers { permanent_uuid: "7205a2fa560f4b448715b67be73c1174" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 43411 } }
I20260811 18:55:38.699376 11386 leader_election.cc:288] T 668822252ffb4af7a2548fda9d4f7b6e P 7205a2fa560f4b448715b67be73c1174 [CANDIDATE]: Term 1 election: Requested vote from peers eb05f4d58b3f470881b5172e1f312e0d (127.27.180.129:38277), 1c9d229dc7694099b581b5c31563108d (127.27.180.130:37907)
I20260811 18:55:38.699698 10758 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "668822252ffb4af7a2548fda9d4f7b6e" candidate_uuid: "7205a2fa560f4b448715b67be73c1174" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb05f4d58b3f470881b5172e1f312e0d"
I20260811 18:55:38.699798 10758 raft_consensus.cc:3037] T 668822252ffb4af7a2548fda9d4f7b6e P eb05f4d58b3f470881b5172e1f312e0d [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:55:38.699753 10995 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "668822252ffb4af7a2548fda9d4f7b6e" candidate_uuid: "7205a2fa560f4b448715b67be73c1174" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c9d229dc7694099b581b5c31563108d"
I20260811 18:55:38.699884 10995 raft_consensus.cc:3037] T 668822252ffb4af7a2548fda9d4f7b6e P 1c9d229dc7694099b581b5c31563108d [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:55:38.700696 10758 raft_consensus.cc:2454] T 668822252ffb4af7a2548fda9d4f7b6e P eb05f4d58b3f470881b5172e1f312e0d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7205a2fa560f4b448715b67be73c1174 in term 1.
I20260811 18:55:38.700752 10995 raft_consensus.cc:2454] T 668822252ffb4af7a2548fda9d4f7b6e P 1c9d229dc7694099b581b5c31563108d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7205a2fa560f4b448715b67be73c1174 in term 1.
I20260811 18:55:38.701018 11122 leader_election.cc:302] T 668822252ffb4af7a2548fda9d4f7b6e P 7205a2fa560f4b448715b67be73c1174 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c9d229dc7694099b581b5c31563108d, 7205a2fa560f4b448715b67be73c1174; no voters:
I20260811 18:55:38.701223 11386 raft_consensus.cc:2781] T 668822252ffb4af7a2548fda9d4f7b6e P 7205a2fa560f4b448715b67be73c1174 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:55:38.701309 11386 raft_consensus.cc:686] T 668822252ffb4af7a2548fda9d4f7b6e P 7205a2fa560f4b448715b67be73c1174 [term 1 LEADER]: Becoming Leader. State: Replica: 7205a2fa560f4b448715b67be73c1174, State: Running, Role: LEADER
I20260811 18:55:38.701450 11386 consensus_queue.cc:237] T 668822252ffb4af7a2548fda9d4f7b6e P 7205a2fa560f4b448715b67be73c1174 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb05f4d58b3f470881b5172e1f312e0d" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38277 } } peers { permanent_uuid: "1c9d229dc7694099b581b5c31563108d" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 37907 } } peers { permanent_uuid: "7205a2fa560f4b448715b67be73c1174" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 43411 } }
I20260811 18:55:38.702154 11430 tablet_replica.cc:439] T 668822252ffb4af7a2548fda9d4f7b6e P 7205a2fa560f4b448715b67be73c1174: TxnStatusTablet state changed. Reason: New leader 7205a2fa560f4b448715b67be73c1174. Latest consensus state: current_term: 1 leader_uuid: "7205a2fa560f4b448715b67be73c1174" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb05f4d58b3f470881b5172e1f312e0d" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38277 } } peers { permanent_uuid: "1c9d229dc7694099b581b5c31563108d" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 37907 } } peers { permanent_uuid: "7205a2fa560f4b448715b67be73c1174" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 43411 } } }
I20260811 18:55:38.702265 11430 tablet_replica.cc:442] T 668822252ffb4af7a2548fda9d4f7b6e P 7205a2fa560f4b448715b67be73c1174: This TxnStatusTablet replica's current role is: LEADER
I20260811 18:55:38.702577 11457 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 18:55:38.702744 11457 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 18:55:38.703189 10462 catalog_manager.cc:5495] T 668822252ffb4af7a2548fda9d4f7b6e P 7205a2fa560f4b448715b67be73c1174 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7205a2fa560f4b448715b67be73c1174 (127.27.180.131). New cstate: current_term: 1 leader_uuid: "7205a2fa560f4b448715b67be73c1174" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb05f4d58b3f470881b5172e1f312e0d" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c9d229dc7694099b581b5c31563108d" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 37907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7205a2fa560f4b448715b67be73c1174" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 43411 } health_report { overall_health: HEALTHY } } }
I20260811 18:55:38.715206 11443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:38.711577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.131:43411 (local address 127.0.0.1:37340)
0811 18:55:38.711711 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:38.711726 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:38.711832 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:38.712135 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:38.712138 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:38.712146 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 18:55:38.712250 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:38.712256 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:38.713000 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:38.713003 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:38.713733 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:38.713744 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:38.714174 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:38.714178 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:38.714269 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:38.714314 (+ 45us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 18:55:38.714922 (+ 608us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 18:55:38.714938 (+ 16us) client_negotiation.cc:769] Sending connection context
0811 18:55:38.715002 (+ 64us) client_negotiation.cc:240] Negotiation successful
0811 18:55:38.715093 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260811 18:55:38.715226 11367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:38.711685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37340 (local address 127.27.180.131:43411)
0811 18:55:38.711807 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:38.711811 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:38.711825 (+ 14us) server_negotiation.cc:407] Connection header received
0811 18:55:38.711974 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:38.711977 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:38.712033 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:38.712111 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:55:38.712350 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:38.712887 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:38.713847 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:38.714076 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:38.714164 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:38.714453 (+ 289us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:55:38.714821 (+ 368us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:55:38.714922 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:38.715013 (+ 91us) server_negotiation.cc:299] Negotiation successful
0811 18:55:38.715066 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 18:55:38.718302 10995 raft_consensus.cc:1261] T 668822252ffb4af7a2548fda9d4f7b6e P 1c9d229dc7694099b581b5c31563108d [term 1 FOLLOWER]: Refusing update from remote peer 7205a2fa560f4b448715b67be73c1174: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:55:38.718515 10758 raft_consensus.cc:1261] T 668822252ffb4af7a2548fda9d4f7b6e P eb05f4d58b3f470881b5172e1f312e0d [term 1 FOLLOWER]: Refusing update from remote peer 7205a2fa560f4b448715b67be73c1174: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:55:38.718868 11430 consensus_queue.cc:1035] T 668822252ffb4af7a2548fda9d4f7b6e P 7205a2fa560f4b448715b67be73c1174 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c9d229dc7694099b581b5c31563108d" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 37907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:55:38.719033 11386 consensus_queue.cc:1035] T 668822252ffb4af7a2548fda9d4f7b6e P 7205a2fa560f4b448715b67be73c1174 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb05f4d58b3f470881b5172e1f312e0d" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:55:38.720476 11387 tablet_replica.cc:439] T 668822252ffb4af7a2548fda9d4f7b6e P 1c9d229dc7694099b581b5c31563108d: TxnStatusTablet state changed. Reason: New leader 7205a2fa560f4b448715b67be73c1174. Latest consensus state: current_term: 1 leader_uuid: "7205a2fa560f4b448715b67be73c1174" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb05f4d58b3f470881b5172e1f312e0d" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38277 } } peers { permanent_uuid: "1c9d229dc7694099b581b5c31563108d" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 37907 } } peers { permanent_uuid: "7205a2fa560f4b448715b67be73c1174" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 43411 } } }
I20260811 18:55:38.720592 11387 tablet_replica.cc:442] T 668822252ffb4af7a2548fda9d4f7b6e P 1c9d229dc7694099b581b5c31563108d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 18:55:38.721582 11393 tablet_replica.cc:439] T 668822252ffb4af7a2548fda9d4f7b6e P eb05f4d58b3f470881b5172e1f312e0d: TxnStatusTablet state changed. Reason: New leader 7205a2fa560f4b448715b67be73c1174. Latest consensus state: current_term: 1 leader_uuid: "7205a2fa560f4b448715b67be73c1174" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb05f4d58b3f470881b5172e1f312e0d" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38277 } } peers { permanent_uuid: "1c9d229dc7694099b581b5c31563108d" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 37907 } } peers { permanent_uuid: "7205a2fa560f4b448715b67be73c1174" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 43411 } } }
I20260811 18:55:38.721698 11393 tablet_replica.cc:442] T 668822252ffb4af7a2548fda9d4f7b6e P eb05f4d58b3f470881b5172e1f312e0d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 18:55:38.723316 11386 tablet_replica.cc:439] T 668822252ffb4af7a2548fda9d4f7b6e P 7205a2fa560f4b448715b67be73c1174: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7205a2fa560f4b448715b67be73c1174" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb05f4d58b3f470881b5172e1f312e0d" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38277 } } peers { permanent_uuid: "1c9d229dc7694099b581b5c31563108d" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 37907 } } peers { permanent_uuid: "7205a2fa560f4b448715b67be73c1174" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 43411 } } }
I20260811 18:55:38.723363 11430 tablet_replica.cc:439] T 668822252ffb4af7a2548fda9d4f7b6e P 7205a2fa560f4b448715b67be73c1174: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7205a2fa560f4b448715b67be73c1174" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb05f4d58b3f470881b5172e1f312e0d" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38277 } } peers { permanent_uuid: "1c9d229dc7694099b581b5c31563108d" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 37907 } } peers { permanent_uuid: "7205a2fa560f4b448715b67be73c1174" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 43411 } } }
I20260811 18:55:38.723469 11430 tablet_replica.cc:442] T 668822252ffb4af7a2548fda9d4f7b6e P 7205a2fa560f4b448715b67be73c1174: This TxnStatusTablet replica's current role is: LEADER
I20260811 18:55:38.723425 11386 tablet_replica.cc:442] T 668822252ffb4af7a2548fda9d4f7b6e P 7205a2fa560f4b448715b67be73c1174: This TxnStatusTablet replica's current role is: LEADER
I20260811 18:55:38.724625 11403 mvcc.cc:205] Tried to move back new op lower bound from 7317399710585323520 to 7317399710522445824. Current Snapshot: MvccSnapshot[applied={T|T < 7317399710585323520}]
I20260811 18:55:38.724805 11387 tablet_replica.cc:439] T 668822252ffb4af7a2548fda9d4f7b6e P 1c9d229dc7694099b581b5c31563108d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7205a2fa560f4b448715b67be73c1174" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb05f4d58b3f470881b5172e1f312e0d" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38277 } } peers { permanent_uuid: "1c9d229dc7694099b581b5c31563108d" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 37907 } } peers { permanent_uuid: "7205a2fa560f4b448715b67be73c1174" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 43411 } } }
I20260811 18:55:38.724895 11387 tablet_replica.cc:442] T 668822252ffb4af7a2548fda9d4f7b6e P 1c9d229dc7694099b581b5c31563108d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 18:55:38.726729 11393 tablet_replica.cc:439] T 668822252ffb4af7a2548fda9d4f7b6e P eb05f4d58b3f470881b5172e1f312e0d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7205a2fa560f4b448715b67be73c1174" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb05f4d58b3f470881b5172e1f312e0d" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38277 } } peers { permanent_uuid: "1c9d229dc7694099b581b5c31563108d" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 37907 } } peers { permanent_uuid: "7205a2fa560f4b448715b67be73c1174" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 43411 } } }
I20260811 18:55:38.726830 11393 tablet_replica.cc:442] T 668822252ffb4af7a2548fda9d4f7b6e P eb05f4d58b3f470881b5172e1f312e0d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 18:55:38.732496 11367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:38.729510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37354 (local address 127.27.180.131:43411)
0811 18:55:38.729615 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:38.729618 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:38.729644 (+ 26us) server_negotiation.cc:407] Connection header received
0811 18:55:38.729766 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:38.729768 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:38.729804 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:38.729854 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:55:38.730075 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:38.730543 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:38.731652 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:38.731847 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:38.731911 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:38.732140 (+ 229us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:55:38.732199 (+ 59us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:55:38.732244 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:38.732341 (+ 97us) server_negotiation.cc:299] Negotiation successful
0811 18:55:38.732380 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 18:55:38.742398 11367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:38.738952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.131:50949 (local address 127.27.180.131:43411)
0811 18:55:38.739043 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:38.739046 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:38.739274 (+ 228us) server_negotiation.cc:407] Connection header received
0811 18:55:38.739396 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:38.739398 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:38.739445 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:38.739502 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:55:38.739750 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:38.740189 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:38.741439 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:38.741698 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:38.741775 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:38.742058 (+ 283us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:55:38.742114 (+ 56us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:55:38.742162 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:38.742250 (+ 88us) server_negotiation.cc:299] Negotiation successful
0811 18:55:38.742306 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260811 18:55:38.742537 11473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:38.738876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.131:43411 (local address 127.27.180.131:50949)
0811 18:55:38.739201 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:38.739214 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:38.739324 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:38.739524 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:38.739527 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:38.739534 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 18:55:38.739658 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:38.739663 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:38.740273 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:38.740276 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:38.741308 (+ 1032us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:38.741319 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:38.741798 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:38.741801 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:38.741899 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:38.741950 (+ 51us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 18:55:38.742178 (+ 228us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 18:55:38.742184 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 18:55:38.742245 (+ 61us) client_negotiation.cc:240] Negotiation successful
0811 18:55:38.742350 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"mutex_wait_us":19,"thread_start_us":90,"threads_started":1}
I20260811 18:55:38.748649 10995 raft_consensus.cc:1261] T adee31878d5647fa90e59d3a6c78cb95 P 1c9d229dc7694099b581b5c31563108d [term 1 FOLLOWER]: Refusing update from remote peer 7205a2fa560f4b448715b67be73c1174: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:55:38.748665 10758 raft_consensus.cc:1261] T adee31878d5647fa90e59d3a6c78cb95 P eb05f4d58b3f470881b5172e1f312e0d [term 1 FOLLOWER]: Refusing update from remote peer 7205a2fa560f4b448715b67be73c1174: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:55:38.749018 11469 consensus_queue.cc:1035] T adee31878d5647fa90e59d3a6c78cb95 P 7205a2fa560f4b448715b67be73c1174 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c9d229dc7694099b581b5c31563108d" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 37907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:55:38.749187 11430 consensus_queue.cc:1035] T adee31878d5647fa90e59d3a6c78cb95 P 7205a2fa560f4b448715b67be73c1174 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb05f4d58b3f470881b5172e1f312e0d" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:55:38.800745 10459 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35620:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:55:38.800886 10459 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35620:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:55:38.802856 10459 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8627c387211147ed82345656961a4525: Sending DeleteTablet for 3 replicas of tablet adee31878d5647fa90e59d3a6c78cb95
I20260811 18:55:38.803339 10721 tablet_service.cc:1514] Processing DeleteTablet for tablet adee31878d5647fa90e59d3a6c78cb95 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:55:38 UTC) from {username='oryx', principal='oryx/127.27.180.190@KRBTEST.COM'} at 127.0.0.1:38594
I20260811 18:55:38.803391 11207 tablet_service.cc:1514] Processing DeleteTablet for tablet adee31878d5647fa90e59d3a6c78cb95 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:55:38 UTC) from {username='oryx', principal='oryx/127.27.180.190@KRBTEST.COM'} at 127.0.0.1:37328
I20260811 18:55:38.803520 11421 tablet_replica.cc:330] T adee31878d5647fa90e59d3a6c78cb95 P eb05f4d58b3f470881b5172e1f312e0d: stopping tablet replica
I20260811 18:55:38.803565 11424 tablet_replica.cc:330] T adee31878d5647fa90e59d3a6c78cb95 P 7205a2fa560f4b448715b67be73c1174: stopping tablet replica
I20260811 18:55:38.803570 10958 tablet_service.cc:1514] Processing DeleteTablet for tablet adee31878d5647fa90e59d3a6c78cb95 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:55:38 UTC) from {username='oryx', principal='oryx/127.27.180.190@KRBTEST.COM'} at 127.0.0.1:57478
I20260811 18:55:38.803632 11421 raft_consensus.cc:2229] T adee31878d5647fa90e59d3a6c78cb95 P eb05f4d58b3f470881b5172e1f312e0d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:55:38.803678 11424 raft_consensus.cc:2229] T adee31878d5647fa90e59d3a6c78cb95 P 7205a2fa560f4b448715b67be73c1174 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:55:38.803740 11421 raft_consensus.cc:2258] T adee31878d5647fa90e59d3a6c78cb95 P eb05f4d58b3f470881b5172e1f312e0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:55:38.803740 11422 tablet_replica.cc:330] T adee31878d5647fa90e59d3a6c78cb95 P 1c9d229dc7694099b581b5c31563108d: stopping tablet replica
I20260811 18:55:38.803848 11422 raft_consensus.cc:2229] T adee31878d5647fa90e59d3a6c78cb95 P 1c9d229dc7694099b581b5c31563108d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:55:38.803944 11424 raft_consensus.cc:2258] T adee31878d5647fa90e59d3a6c78cb95 P 7205a2fa560f4b448715b67be73c1174 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:55:38.803954 11422 raft_consensus.cc:2258] T adee31878d5647fa90e59d3a6c78cb95 P 1c9d229dc7694099b581b5c31563108d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:55:38.804569 11421 ts_tablet_manager.cc:1905] T adee31878d5647fa90e59d3a6c78cb95 P eb05f4d58b3f470881b5172e1f312e0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:55:38.804651 11424 ts_tablet_manager.cc:1905] T adee31878d5647fa90e59d3a6c78cb95 P 7205a2fa560f4b448715b67be73c1174: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:55:38.806912 11421 ts_tablet_manager.cc:1918] T adee31878d5647fa90e59d3a6c78cb95 P eb05f4d58b3f470881b5172e1f312e0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 18:55:38.806980 11421 log.cc:1196] T adee31878d5647fa90e59d3a6c78cb95 P eb05f4d58b3f470881b5172e1f312e0d: Deleting WAL directory at /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/minicluster-data/ts-0/wal/wals/adee31878d5647fa90e59d3a6c78cb95
I20260811 18:55:38.807197 11421 ts_tablet_manager.cc:1939] T adee31878d5647fa90e59d3a6c78cb95 P eb05f4d58b3f470881b5172e1f312e0d: Deleting consensus metadata
I20260811 18:55:38.807485 11424 ts_tablet_manager.cc:1918] T adee31878d5647fa90e59d3a6c78cb95 P 7205a2fa560f4b448715b67be73c1174: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 18:55:38.807564 11424 log.cc:1196] T adee31878d5647fa90e59d3a6c78cb95 P 7205a2fa560f4b448715b67be73c1174: Deleting WAL directory at /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/minicluster-data/ts-2/wal/wals/adee31878d5647fa90e59d3a6c78cb95
I20260811 18:55:38.807806 11424 ts_tablet_manager.cc:1939] T adee31878d5647fa90e59d3a6c78cb95 P 7205a2fa560f4b448715b67be73c1174: Deleting consensus metadata
I20260811 18:55:38.807776 10430 catalog_manager.cc:4841] TS eb05f4d58b3f470881b5172e1f312e0d (127.27.180.129:38277): tablet adee31878d5647fa90e59d3a6c78cb95 (table test-table [id=ba3fbc7c760e41ad80cc7c9d98a29499]) successfully deleted
I20260811 18:55:38.808394 10430 catalog_manager.cc:4841] TS 7205a2fa560f4b448715b67be73c1174 (127.27.180.131:43411): tablet adee31878d5647fa90e59d3a6c78cb95 (table test-table [id=ba3fbc7c760e41ad80cc7c9d98a29499]) successfully deleted
I20260811 18:55:38.812108 11422 ts_tablet_manager.cc:1905] T adee31878d5647fa90e59d3a6c78cb95 P 1c9d229dc7694099b581b5c31563108d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:55:38.814775 11422 ts_tablet_manager.cc:1918] T adee31878d5647fa90e59d3a6c78cb95 P 1c9d229dc7694099b581b5c31563108d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 18:55:38.814845 11422 log.cc:1196] T adee31878d5647fa90e59d3a6c78cb95 P 1c9d229dc7694099b581b5c31563108d: Deleting WAL directory at /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/minicluster-data/ts-1/wal/wals/adee31878d5647fa90e59d3a6c78cb95
I20260811 18:55:38.815052 11422 ts_tablet_manager.cc:1939] T adee31878d5647fa90e59d3a6c78cb95 P 1c9d229dc7694099b581b5c31563108d: Deleting consensus metadata
I20260811 18:55:38.815677 10436 catalog_manager.cc:4841] TS 1c9d229dc7694099b581b5c31563108d (127.27.180.130:37907): tablet adee31878d5647fa90e59d3a6c78cb95 (table test-table [id=ba3fbc7c760e41ad80cc7c9d98a29499]) successfully deleted
Aug 11 18:55:38 dist-test-slave-2j7r krb5kdc[10385](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474538, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 18:55:38 dist-test-slave-2j7r krb5kdc[10385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:55:38 dist-test-slave-2j7r krb5kdc[10385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
W20260811 18:55:38.834188 11450 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:55:38.827509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35632 (local address 127.27.180.190:37195)
0811 18:55:38.827661 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:38.827666 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:38.827836 (+ 170us) server_negotiation.cc:407] Connection header received
0811 18:55:38.827984 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:38.827987 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:38.828035 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:38.828102 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:38.828930 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:38.829502 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:38.830644 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:38.830858 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:38.830934 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:38.834060 (+ 3126us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35632: BlockingRecv error: recv got EOF from 127.0.0.1:35632 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 11 18:55:38 dist-test-slave-2j7r krb5kdc[10385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474538, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM
I20260811 18:55:38.843855 11450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:38.835750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35638 (local address 127.27.180.190:37195)
0811 18:55:38.835859 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:38.835863 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:38.835881 (+ 18us) server_negotiation.cc:407] Connection header received
0811 18:55:38.835978 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:38.835981 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:38.836030 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:38.836118 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:38.836619 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:38.837398 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:38.838181 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:38.838519 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:38.838633 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:38.840836 (+ 2203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:38.840840 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:38.840843 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:38.840869 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:38.842349 (+ 1480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:38.842713 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:38.842716 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:38.842718 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:38.842808 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:38.843033 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:38.843036 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:38.843038 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:38.843268 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:38.843408 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:38.843579 (+ 171us) server_negotiation.cc:299] Negotiation successful
0811 18:55:38.843698 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 18:55:38.846287 10459 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35638:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:55:38.846731 10459 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:55:38.851416 10721 tablet_service.cc:1467] Processing CreateTablet for tablet d68547eda3ed46a1847b1e14cfd14f16 (DEFAULT_TABLE table=test-table [id=cf571680a584474cb76b0a0200576e95]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:55:38.851560 11207 tablet_service.cc:1467] Processing CreateTablet for tablet d68547eda3ed46a1847b1e14cfd14f16 (DEFAULT_TABLE table=test-table [id=cf571680a584474cb76b0a0200576e95]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:55:38.851598 10958 tablet_service.cc:1467] Processing CreateTablet for tablet d68547eda3ed46a1847b1e14cfd14f16 (DEFAULT_TABLE table=test-table [id=cf571680a584474cb76b0a0200576e95]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:55:38.851763 10721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d68547eda3ed46a1847b1e14cfd14f16. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:55:38.851820 11207 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d68547eda3ed46a1847b1e14cfd14f16. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:55:38.851888 10958 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d68547eda3ed46a1847b1e14cfd14f16. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:55:38.854212 11372 tablet_bootstrap.cc:492] T d68547eda3ed46a1847b1e14cfd14f16 P 1c9d229dc7694099b581b5c31563108d: Bootstrap starting.
I20260811 18:55:38.854328 11373 tablet_bootstrap.cc:492] T d68547eda3ed46a1847b1e14cfd14f16 P eb05f4d58b3f470881b5172e1f312e0d: Bootstrap starting.
I20260811 18:55:38.854732 11371 tablet_bootstrap.cc:492] T d68547eda3ed46a1847b1e14cfd14f16 P 7205a2fa560f4b448715b67be73c1174: Bootstrap starting.
I20260811 18:55:38.855070 11372 tablet_bootstrap.cc:654] T d68547eda3ed46a1847b1e14cfd14f16 P 1c9d229dc7694099b581b5c31563108d: Neither blocks nor log segments found. Creating new log.
I20260811 18:55:38.855170 11373 tablet_bootstrap.cc:654] T d68547eda3ed46a1847b1e14cfd14f16 P eb05f4d58b3f470881b5172e1f312e0d: Neither blocks nor log segments found. Creating new log.
I20260811 18:55:38.856060 11371 tablet_bootstrap.cc:654] T d68547eda3ed46a1847b1e14cfd14f16 P 7205a2fa560f4b448715b67be73c1174: Neither blocks nor log segments found. Creating new log.
I20260811 18:55:38.856492 11372 tablet_bootstrap.cc:492] T d68547eda3ed46a1847b1e14cfd14f16 P 1c9d229dc7694099b581b5c31563108d: No bootstrap required, opened a new log
I20260811 18:55:38.856573 11372 ts_tablet_manager.cc:1397] T d68547eda3ed46a1847b1e14cfd14f16 P 1c9d229dc7694099b581b5c31563108d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 18:55:38.857211 11372 raft_consensus.cc:348] T d68547eda3ed46a1847b1e14cfd14f16 P 1c9d229dc7694099b581b5c31563108d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb05f4d58b3f470881b5172e1f312e0d" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38277 } } peers { permanent_uuid: "1c9d229dc7694099b581b5c31563108d" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 37907 } } peers { permanent_uuid: "7205a2fa560f4b448715b67be73c1174" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 43411 } }
I20260811 18:55:38.857307 11373 tablet_bootstrap.cc:492] T d68547eda3ed46a1847b1e14cfd14f16 P eb05f4d58b3f470881b5172e1f312e0d: No bootstrap required, opened a new log
I20260811 18:55:38.857322 11372 raft_consensus.cc:374] T d68547eda3ed46a1847b1e14cfd14f16 P 1c9d229dc7694099b581b5c31563108d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:55:38.857352 11372 raft_consensus.cc:729] T d68547eda3ed46a1847b1e14cfd14f16 P 1c9d229dc7694099b581b5c31563108d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c9d229dc7694099b581b5c31563108d, State: Initialized, Role: FOLLOWER
I20260811 18:55:38.857386 11373 ts_tablet_manager.cc:1397] T d68547eda3ed46a1847b1e14cfd14f16 P eb05f4d58b3f470881b5172e1f312e0d: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 18:55:38.857456 11372 consensus_queue.cc:260] T d68547eda3ed46a1847b1e14cfd14f16 P 1c9d229dc7694099b581b5c31563108d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb05f4d58b3f470881b5172e1f312e0d" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38277 } } peers { permanent_uuid: "1c9d229dc7694099b581b5c31563108d" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 37907 } } peers { permanent_uuid: "7205a2fa560f4b448715b67be73c1174" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 43411 } }
I20260811 18:55:38.857693 11372 ts_tablet_manager.cc:1428] T d68547eda3ed46a1847b1e14cfd14f16 P 1c9d229dc7694099b581b5c31563108d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 18:55:38.857750 11373 raft_consensus.cc:348] T d68547eda3ed46a1847b1e14cfd14f16 P eb05f4d58b3f470881b5172e1f312e0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb05f4d58b3f470881b5172e1f312e0d" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38277 } } peers { permanent_uuid: "1c9d229dc7694099b581b5c31563108d" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 37907 } } peers { permanent_uuid: "7205a2fa560f4b448715b67be73c1174" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 43411 } }
I20260811 18:55:38.857856 11373 raft_consensus.cc:374] T d68547eda3ed46a1847b1e14cfd14f16 P eb05f4d58b3f470881b5172e1f312e0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:55:38.857919 11373 raft_consensus.cc:729] T d68547eda3ed46a1847b1e14cfd14f16 P eb05f4d58b3f470881b5172e1f312e0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb05f4d58b3f470881b5172e1f312e0d, State: Initialized, Role: FOLLOWER
I20260811 18:55:38.858036 11373 consensus_queue.cc:260] T d68547eda3ed46a1847b1e14cfd14f16 P eb05f4d58b3f470881b5172e1f312e0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb05f4d58b3f470881b5172e1f312e0d" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38277 } } peers { permanent_uuid: "1c9d229dc7694099b581b5c31563108d" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 37907 } } peers { permanent_uuid: "7205a2fa560f4b448715b67be73c1174" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 43411 } }
I20260811 18:55:38.858291 11373 ts_tablet_manager.cc:1428] T d68547eda3ed46a1847b1e14cfd14f16 P eb05f4d58b3f470881b5172e1f312e0d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 18:55:38.861877 11371 tablet_bootstrap.cc:492] T d68547eda3ed46a1847b1e14cfd14f16 P 7205a2fa560f4b448715b67be73c1174: No bootstrap required, opened a new log
I20260811 18:55:38.861979 11371 ts_tablet_manager.cc:1397] T d68547eda3ed46a1847b1e14cfd14f16 P 7205a2fa560f4b448715b67be73c1174: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.000s
I20260811 18:55:38.862381 11371 raft_consensus.cc:348] T d68547eda3ed46a1847b1e14cfd14f16 P 7205a2fa560f4b448715b67be73c1174 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb05f4d58b3f470881b5172e1f312e0d" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38277 } } peers { permanent_uuid: "1c9d229dc7694099b581b5c31563108d" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 37907 } } peers { permanent_uuid: "7205a2fa560f4b448715b67be73c1174" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 43411 } }
I20260811 18:55:38.862485 11371 raft_consensus.cc:374] T d68547eda3ed46a1847b1e14cfd14f16 P 7205a2fa560f4b448715b67be73c1174 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:55:38.862524 11371 raft_consensus.cc:729] T d68547eda3ed46a1847b1e14cfd14f16 P 7205a2fa560f4b448715b67be73c1174 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7205a2fa560f4b448715b67be73c1174, State: Initialized, Role: FOLLOWER
I20260811 18:55:38.862743 11371 consensus_queue.cc:260] T d68547eda3ed46a1847b1e14cfd14f16 P 7205a2fa560f4b448715b67be73c1174 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb05f4d58b3f470881b5172e1f312e0d" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38277 } } peers { permanent_uuid: "1c9d229dc7694099b581b5c31563108d" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 37907 } } peers { permanent_uuid: "7205a2fa560f4b448715b67be73c1174" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 43411 } }
I20260811 18:55:38.863111 11371 ts_tablet_manager.cc:1428] T d68547eda3ed46a1847b1e14cfd14f16 P 7205a2fa560f4b448715b67be73c1174: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 18:55:39.195838 11387 raft_consensus.cc:482] T d68547eda3ed46a1847b1e14cfd14f16 P 1c9d229dc7694099b581b5c31563108d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:55:39.196012 11387 raft_consensus.cc:504] T d68547eda3ed46a1847b1e14cfd14f16 P 1c9d229dc7694099b581b5c31563108d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb05f4d58b3f470881b5172e1f312e0d" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38277 } } peers { permanent_uuid: "1c9d229dc7694099b581b5c31563108d" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 37907 } } peers { permanent_uuid: "7205a2fa560f4b448715b67be73c1174" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 43411 } }
I20260811 18:55:39.197216 11387 leader_election.cc:288] T d68547eda3ed46a1847b1e14cfd14f16 P 1c9d229dc7694099b581b5c31563108d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb05f4d58b3f470881b5172e1f312e0d (127.27.180.129:38277), 7205a2fa560f4b448715b67be73c1174 (127.27.180.131:43411)
I20260811 18:55:39.201438 11367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:39.197995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.130:56655 (local address 127.27.180.131:43411)
0811 18:55:39.198104 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:39.198108 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:39.198257 (+ 149us) server_negotiation.cc:407] Connection header received
0811 18:55:39.198374 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:39.198385 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:39.198431 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:39.198514 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:55:39.198769 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:39.199246 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:39.200371 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:39.200777 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:39.200886 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:39.200944 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:39.201238 (+ 294us) server_negotiation.cc:299] Negotiation successful
0811 18:55:39.201307 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260811 18:55:39.201448 11515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:39.197591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.130:43169 (local address 127.27.180.129:38277)
0811 18:55:39.197825 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:39.197831 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:39.197850 (+ 19us) server_negotiation.cc:407] Connection header received
0811 18:55:39.198045 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:39.198048 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:39.198106 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:39.198194 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:55:39.198545 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:39.199201 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:39.200358 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:39.200779 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:39.200891 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:39.200956 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:39.201235 (+ 279us) server_negotiation.cc:299] Negotiation successful
0811 18:55:39.201319 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":70,"threads_started":1}
I20260811 18:55:39.201471 11514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:39.197499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.129:38277 (local address 127.27.180.130:43169)
0811 18:55:39.197743 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:39.197758 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:39.197875 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:39.198231 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:39.198235 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:39.198263 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:55:39.198411 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:39.198418 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:39.199323 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:39.199327 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:39.200231 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:39.200241 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:39.200910 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:39.200914 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:39.201072 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:39.201130 (+ 58us) client_negotiation.cc:769] Sending connection context
0811 18:55:39.201202 (+ 72us) client_negotiation.cc:240] Negotiation successful
0811 18:55:39.201282 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"mutex_wait_us":73,"spinlock_wait_cycles":768,"thread_start_us":69,"threads_started":1}
I20260811 18:55:39.201476 11516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:39.197884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.131:43411 (local address 127.27.180.130:56655)
0811 18:55:39.198194 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:39.198206 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:39.198292 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:39.198530 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:39.198532 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:39.198549 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:55:39.198667 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:39.198673 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:39.199358 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:39.199361 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:39.200242 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:39.200252 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:39.200890 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:39.200894 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:39.201072 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:39.201143 (+ 71us) client_negotiation.cc:769] Sending connection context
0811 18:55:39.201218 (+ 75us) client_negotiation.cc:240] Negotiation successful
0811 18:55:39.201289 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"mutex_wait_us":96,"thread_start_us":72,"threads_started":1}
I20260811 18:55:39.202013 11238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d68547eda3ed46a1847b1e14cfd14f16" candidate_uuid: "1c9d229dc7694099b581b5c31563108d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7205a2fa560f4b448715b67be73c1174" is_pre_election: true
I20260811 18:55:39.202122 10758 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d68547eda3ed46a1847b1e14cfd14f16" candidate_uuid: "1c9d229dc7694099b581b5c31563108d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb05f4d58b3f470881b5172e1f312e0d" is_pre_election: true
I20260811 18:55:39.202172 11238 raft_consensus.cc:2454] T d68547eda3ed46a1847b1e14cfd14f16 P 7205a2fa560f4b448715b67be73c1174 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1c9d229dc7694099b581b5c31563108d in term 0.
I20260811 18:55:39.202271 10758 raft_consensus.cc:2454] T d68547eda3ed46a1847b1e14cfd14f16 P eb05f4d58b3f470881b5172e1f312e0d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1c9d229dc7694099b581b5c31563108d in term 0.
I20260811 18:55:39.202505 10875 leader_election.cc:302] T d68547eda3ed46a1847b1e14cfd14f16 P 1c9d229dc7694099b581b5c31563108d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c9d229dc7694099b581b5c31563108d, 7205a2fa560f4b448715b67be73c1174; no voters:
I20260811 18:55:39.202744 11387 raft_consensus.cc:2781] T d68547eda3ed46a1847b1e14cfd14f16 P 1c9d229dc7694099b581b5c31563108d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:55:39.202801 11387 raft_consensus.cc:482] T d68547eda3ed46a1847b1e14cfd14f16 P 1c9d229dc7694099b581b5c31563108d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:55:39.202841 11387 raft_consensus.cc:3037] T d68547eda3ed46a1847b1e14cfd14f16 P 1c9d229dc7694099b581b5c31563108d [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:55:39.203819 11387 raft_consensus.cc:504] T d68547eda3ed46a1847b1e14cfd14f16 P 1c9d229dc7694099b581b5c31563108d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb05f4d58b3f470881b5172e1f312e0d" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38277 } } peers { permanent_uuid: "1c9d229dc7694099b581b5c31563108d" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 37907 } } peers { permanent_uuid: "7205a2fa560f4b448715b67be73c1174" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 43411 } }
I20260811 18:55:39.204150 11387 leader_election.cc:288] T d68547eda3ed46a1847b1e14cfd14f16 P 1c9d229dc7694099b581b5c31563108d [CANDIDATE]: Term 1 election: Requested vote from peers eb05f4d58b3f470881b5172e1f312e0d (127.27.180.129:38277), 7205a2fa560f4b448715b67be73c1174 (127.27.180.131:43411)
I20260811 18:55:39.204476 10758 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d68547eda3ed46a1847b1e14cfd14f16" candidate_uuid: "1c9d229dc7694099b581b5c31563108d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb05f4d58b3f470881b5172e1f312e0d"
I20260811 18:55:39.204557 11238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d68547eda3ed46a1847b1e14cfd14f16" candidate_uuid: "1c9d229dc7694099b581b5c31563108d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7205a2fa560f4b448715b67be73c1174"
I20260811 18:55:39.204594 10758 raft_consensus.cc:3037] T d68547eda3ed46a1847b1e14cfd14f16 P eb05f4d58b3f470881b5172e1f312e0d [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:55:39.204680 11238 raft_consensus.cc:3037] T d68547eda3ed46a1847b1e14cfd14f16 P 7205a2fa560f4b448715b67be73c1174 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:55:39.205572 10758 raft_consensus.cc:2454] T d68547eda3ed46a1847b1e14cfd14f16 P eb05f4d58b3f470881b5172e1f312e0d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c9d229dc7694099b581b5c31563108d in term 1.
I20260811 18:55:39.205663 11238 raft_consensus.cc:2454] T d68547eda3ed46a1847b1e14cfd14f16 P 7205a2fa560f4b448715b67be73c1174 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c9d229dc7694099b581b5c31563108d in term 1.
I20260811 18:55:39.205853 10875 leader_election.cc:302] T d68547eda3ed46a1847b1e14cfd14f16 P 1c9d229dc7694099b581b5c31563108d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c9d229dc7694099b581b5c31563108d, eb05f4d58b3f470881b5172e1f312e0d; no voters:
I20260811 18:55:39.206041 11387 raft_consensus.cc:2781] T d68547eda3ed46a1847b1e14cfd14f16 P 1c9d229dc7694099b581b5c31563108d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:55:39.206212 11387 raft_consensus.cc:686] T d68547eda3ed46a1847b1e14cfd14f16 P 1c9d229dc7694099b581b5c31563108d [term 1 LEADER]: Becoming Leader. State: Replica: 1c9d229dc7694099b581b5c31563108d, State: Running, Role: LEADER
I20260811 18:55:39.206466 11387 consensus_queue.cc:237] T d68547eda3ed46a1847b1e14cfd14f16 P 1c9d229dc7694099b581b5c31563108d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb05f4d58b3f470881b5172e1f312e0d" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38277 } } peers { permanent_uuid: "1c9d229dc7694099b581b5c31563108d" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 37907 } } peers { permanent_uuid: "7205a2fa560f4b448715b67be73c1174" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 43411 } }
I20260811 18:55:39.208761 10460 catalog_manager.cc:5495] T d68547eda3ed46a1847b1e14cfd14f16 P 1c9d229dc7694099b581b5c31563108d reported cstate change: term changed from 0 to 1, leader changed from <none> to 1c9d229dc7694099b581b5c31563108d (127.27.180.130). New cstate: current_term: 1 leader_uuid: "1c9d229dc7694099b581b5c31563108d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb05f4d58b3f470881b5172e1f312e0d" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c9d229dc7694099b581b5c31563108d" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 37907 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7205a2fa560f4b448715b67be73c1174" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 43411 } health_report { overall_health: UNKNOWN } } }
I20260811 18:55:39.225785 11524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:39.222239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57482 (local address 127.27.180.130:37907)
0811 18:55:39.222548 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:39.222555 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:39.222568 (+ 13us) server_negotiation.cc:407] Connection header received
0811 18:55:39.222641 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:39.222643 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:39.222697 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:39.222799 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:55:39.223002 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:39.223531 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:39.224406 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:39.224653 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:39.224729 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:39.224916 (+ 187us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:55:39.225426 (+ 510us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:55:39.225515 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:39.225576 (+ 61us) server_negotiation.cc:299] Negotiation successful
0811 18:55:39.225623 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":240,"thread_start_us":150,"threads_started":1}
I20260811 18:55:39.228380 10758 raft_consensus.cc:1261] T d68547eda3ed46a1847b1e14cfd14f16 P eb05f4d58b3f470881b5172e1f312e0d [term 1 FOLLOWER]: Refusing update from remote peer 1c9d229dc7694099b581b5c31563108d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:55:39.228540 11238 raft_consensus.cc:1261] T d68547eda3ed46a1847b1e14cfd14f16 P 7205a2fa560f4b448715b67be73c1174 [term 1 FOLLOWER]: Refusing update from remote peer 1c9d229dc7694099b581b5c31563108d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:55:39.228857 11518 consensus_queue.cc:1035] T d68547eda3ed46a1847b1e14cfd14f16 P 1c9d229dc7694099b581b5c31563108d [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb05f4d58b3f470881b5172e1f312e0d" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:55:39.229022 11387 consensus_queue.cc:1035] T d68547eda3ed46a1847b1e14cfd14f16 P 1c9d229dc7694099b581b5c31563108d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7205a2fa560f4b448715b67be73c1174" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 43411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:55:39.239691 10462 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35638:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:55:39.239869 10462 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35638:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:55:39.241984 10462 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8627c387211147ed82345656961a4525: Sending DeleteTablet for 3 replicas of tablet d68547eda3ed46a1847b1e14cfd14f16
I20260811 18:55:39.242611 10958 tablet_service.cc:1514] Processing DeleteTablet for tablet d68547eda3ed46a1847b1e14cfd14f16 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:55:39 UTC) from {username='oryx', principal='oryx/127.27.180.190@KRBTEST.COM'} at 127.0.0.1:57478
I20260811 18:55:39.242631 11207 tablet_service.cc:1514] Processing DeleteTablet for tablet d68547eda3ed46a1847b1e14cfd14f16 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:55:39 UTC) from {username='oryx', principal='oryx/127.27.180.190@KRBTEST.COM'} at 127.0.0.1:37328
I20260811 18:55:39.242816 10721 tablet_service.cc:1514] Processing DeleteTablet for tablet d68547eda3ed46a1847b1e14cfd14f16 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:55:39 UTC) from {username='oryx', principal='oryx/127.27.180.190@KRBTEST.COM'} at 127.0.0.1:38594
I20260811 18:55:39.242910 11422 tablet_replica.cc:330] T d68547eda3ed46a1847b1e14cfd14f16 P 1c9d229dc7694099b581b5c31563108d: stopping tablet replica
I20260811 18:55:39.242995 11424 tablet_replica.cc:330] T d68547eda3ed46a1847b1e14cfd14f16 P 7205a2fa560f4b448715b67be73c1174: stopping tablet replica
I20260811 18:55:39.243007 11421 tablet_replica.cc:330] T d68547eda3ed46a1847b1e14cfd14f16 P eb05f4d58b3f470881b5172e1f312e0d: stopping tablet replica
I20260811 18:55:39.243040 11422 raft_consensus.cc:2229] T d68547eda3ed46a1847b1e14cfd14f16 P 1c9d229dc7694099b581b5c31563108d [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:55:39.243131 11421 raft_consensus.cc:2229] T d68547eda3ed46a1847b1e14cfd14f16 P eb05f4d58b3f470881b5172e1f312e0d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:55:39.243131 11424 raft_consensus.cc:2229] T d68547eda3ed46a1847b1e14cfd14f16 P 7205a2fa560f4b448715b67be73c1174 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:55:39.243271 11421 raft_consensus.cc:2258] T d68547eda3ed46a1847b1e14cfd14f16 P eb05f4d58b3f470881b5172e1f312e0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:55:39.243275 11424 raft_consensus.cc:2258] T d68547eda3ed46a1847b1e14cfd14f16 P 7205a2fa560f4b448715b67be73c1174 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:55:39.243328 11422 raft_consensus.cc:2258] T d68547eda3ed46a1847b1e14cfd14f16 P 1c9d229dc7694099b581b5c31563108d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:55:39.243999 11421 ts_tablet_manager.cc:1905] T d68547eda3ed46a1847b1e14cfd14f16 P eb05f4d58b3f470881b5172e1f312e0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:55:39.244136 11424 ts_tablet_manager.cc:1905] T d68547eda3ed46a1847b1e14cfd14f16 P 7205a2fa560f4b448715b67be73c1174: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:55:39.244134 10462 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35638:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:55:39.244536 10462 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:55:39.246086 11422 ts_tablet_manager.cc:1905] T d68547eda3ed46a1847b1e14cfd14f16 P 1c9d229dc7694099b581b5c31563108d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:55:39.246596 11424 ts_tablet_manager.cc:1918] T d68547eda3ed46a1847b1e14cfd14f16 P 7205a2fa560f4b448715b67be73c1174: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:55:39.246659 11424 log.cc:1196] T d68547eda3ed46a1847b1e14cfd14f16 P 7205a2fa560f4b448715b67be73c1174: Deleting WAL directory at /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/minicluster-data/ts-2/wal/wals/d68547eda3ed46a1847b1e14cfd14f16
I20260811 18:55:39.246661 11421 ts_tablet_manager.cc:1918] T d68547eda3ed46a1847b1e14cfd14f16 P eb05f4d58b3f470881b5172e1f312e0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:55:39.246750 11421 log.cc:1196] T d68547eda3ed46a1847b1e14cfd14f16 P eb05f4d58b3f470881b5172e1f312e0d: Deleting WAL directory at /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/minicluster-data/ts-0/wal/wals/d68547eda3ed46a1847b1e14cfd14f16
I20260811 18:55:39.246883 11424 ts_tablet_manager.cc:1939] T d68547eda3ed46a1847b1e14cfd14f16 P 7205a2fa560f4b448715b67be73c1174: Deleting consensus metadata
I20260811 18:55:39.247026 11421 ts_tablet_manager.cc:1939] T d68547eda3ed46a1847b1e14cfd14f16 P eb05f4d58b3f470881b5172e1f312e0d: Deleting consensus metadata
I20260811 18:55:39.247684 10430 catalog_manager.cc:4841] TS 7205a2fa560f4b448715b67be73c1174 (127.27.180.131:43411): tablet d68547eda3ed46a1847b1e14cfd14f16 (table test-table [id=cf571680a584474cb76b0a0200576e95]) successfully deleted
I20260811 18:55:39.248373 11422 ts_tablet_manager.cc:1918] T d68547eda3ed46a1847b1e14cfd14f16 P 1c9d229dc7694099b581b5c31563108d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:55:39.248463 11422 log.cc:1196] T d68547eda3ed46a1847b1e14cfd14f16 P 1c9d229dc7694099b581b5c31563108d: Deleting WAL directory at /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/minicluster-data/ts-1/wal/wals/d68547eda3ed46a1847b1e14cfd14f16
I20260811 18:55:39.248692 11422 ts_tablet_manager.cc:1939] T d68547eda3ed46a1847b1e14cfd14f16 P 1c9d229dc7694099b581b5c31563108d: Deleting consensus metadata
I20260811 18:55:39.248656 10430 catalog_manager.cc:4841] TS eb05f4d58b3f470881b5172e1f312e0d (127.27.180.129:38277): tablet d68547eda3ed46a1847b1e14cfd14f16 (table test-table [id=cf571680a584474cb76b0a0200576e95]) successfully deleted
I20260811 18:55:39.249591 10436 catalog_manager.cc:4841] TS 1c9d229dc7694099b581b5c31563108d (127.27.180.130:37907): tablet d68547eda3ed46a1847b1e14cfd14f16 (table test-table [id=cf571680a584474cb76b0a0200576e95]) successfully deleted
I20260811 18:55:39.251758 10958 tablet_service.cc:1467] Processing CreateTablet for tablet 685346c2a68e44cc8c1fbc3b8905c2b5 (DEFAULT_TABLE table=test-table [id=ea384bf831174be5bcf41124bb5b1f35]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:55:39.251748 10721 tablet_service.cc:1467] Processing CreateTablet for tablet 685346c2a68e44cc8c1fbc3b8905c2b5 (DEFAULT_TABLE table=test-table [id=ea384bf831174be5bcf41124bb5b1f35]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:55:39.251883 11207 tablet_service.cc:1467] Processing CreateTablet for tablet 685346c2a68e44cc8c1fbc3b8905c2b5 (DEFAULT_TABLE table=test-table [id=ea384bf831174be5bcf41124bb5b1f35]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:55:39.252029 10958 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 685346c2a68e44cc8c1fbc3b8905c2b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:55:39.252030 10721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 685346c2a68e44cc8c1fbc3b8905c2b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:55:39.252110 11207 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 685346c2a68e44cc8c1fbc3b8905c2b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:55:39.254318 11372 tablet_bootstrap.cc:492] T 685346c2a68e44cc8c1fbc3b8905c2b5 P 1c9d229dc7694099b581b5c31563108d: Bootstrap starting.
I20260811 18:55:39.254318 11371 tablet_bootstrap.cc:492] T 685346c2a68e44cc8c1fbc3b8905c2b5 P 7205a2fa560f4b448715b67be73c1174: Bootstrap starting.
I20260811 18:55:39.254588 11373 tablet_bootstrap.cc:492] T 685346c2a68e44cc8c1fbc3b8905c2b5 P eb05f4d58b3f470881b5172e1f312e0d: Bootstrap starting.
I20260811 18:55:39.255131 11371 tablet_bootstrap.cc:654] T 685346c2a68e44cc8c1fbc3b8905c2b5 P 7205a2fa560f4b448715b67be73c1174: Neither blocks nor log segments found. Creating new log.
I20260811 18:55:39.255188 11372 tablet_bootstrap.cc:654] T 685346c2a68e44cc8c1fbc3b8905c2b5 P 1c9d229dc7694099b581b5c31563108d: Neither blocks nor log segments found. Creating new log.
I20260811 18:55:39.255437 11373 tablet_bootstrap.cc:654] T 685346c2a68e44cc8c1fbc3b8905c2b5 P eb05f4d58b3f470881b5172e1f312e0d: Neither blocks nor log segments found. Creating new log.
I20260811 18:55:39.256669 11371 tablet_bootstrap.cc:492] T 685346c2a68e44cc8c1fbc3b8905c2b5 P 7205a2fa560f4b448715b67be73c1174: No bootstrap required, opened a new log
I20260811 18:55:39.256757 11371 ts_tablet_manager.cc:1397] T 685346c2a68e44cc8c1fbc3b8905c2b5 P 7205a2fa560f4b448715b67be73c1174: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 18:55:39.257051 11373 tablet_bootstrap.cc:492] T 685346c2a68e44cc8c1fbc3b8905c2b5 P eb05f4d58b3f470881b5172e1f312e0d: No bootstrap required, opened a new log
I20260811 18:55:39.257161 11373 ts_tablet_manager.cc:1397] T 685346c2a68e44cc8c1fbc3b8905c2b5 P eb05f4d58b3f470881b5172e1f312e0d: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 18:55:39.257202 11371 raft_consensus.cc:348] T 685346c2a68e44cc8c1fbc3b8905c2b5 P 7205a2fa560f4b448715b67be73c1174 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb05f4d58b3f470881b5172e1f312e0d" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38277 } } peers { permanent_uuid: "1c9d229dc7694099b581b5c31563108d" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 37907 } } peers { permanent_uuid: "7205a2fa560f4b448715b67be73c1174" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 43411 } }
I20260811 18:55:39.257318 11371 raft_consensus.cc:374] T 685346c2a68e44cc8c1fbc3b8905c2b5 P 7205a2fa560f4b448715b67be73c1174 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:55:39.257350 11371 raft_consensus.cc:729] T 685346c2a68e44cc8c1fbc3b8905c2b5 P 7205a2fa560f4b448715b67be73c1174 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7205a2fa560f4b448715b67be73c1174, State: Initialized, Role: FOLLOWER
I20260811 18:55:39.257542 11372 tablet_bootstrap.cc:492] T 685346c2a68e44cc8c1fbc3b8905c2b5 P 1c9d229dc7694099b581b5c31563108d: No bootstrap required, opened a new log
I20260811 18:55:39.257490 11371 consensus_queue.cc:260] T 685346c2a68e44cc8c1fbc3b8905c2b5 P 7205a2fa560f4b448715b67be73c1174 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb05f4d58b3f470881b5172e1f312e0d" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38277 } } peers { permanent_uuid: "1c9d229dc7694099b581b5c31563108d" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 37907 } } peers { permanent_uuid: "7205a2fa560f4b448715b67be73c1174" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 43411 } }
I20260811 18:55:39.257611 11372 ts_tablet_manager.cc:1397] T 685346c2a68e44cc8c1fbc3b8905c2b5 P 1c9d229dc7694099b581b5c31563108d: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260811 18:55:39.257562 11373 raft_consensus.cc:348] T 685346c2a68e44cc8c1fbc3b8905c2b5 P eb05f4d58b3f470881b5172e1f312e0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb05f4d58b3f470881b5172e1f312e0d" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38277 } } peers { permanent_uuid: "1c9d229dc7694099b581b5c31563108d" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 37907 } } peers { permanent_uuid: "7205a2fa560f4b448715b67be73c1174" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 43411 } }
I20260811 18:55:39.257683 11373 raft_consensus.cc:374] T 685346c2a68e44cc8c1fbc3b8905c2b5 P eb05f4d58b3f470881b5172e1f312e0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:55:39.257720 11373 raft_consensus.cc:729] T 685346c2a68e44cc8c1fbc3b8905c2b5 P eb05f4d58b3f470881b5172e1f312e0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb05f4d58b3f470881b5172e1f312e0d, State: Initialized, Role: FOLLOWER
I20260811 18:55:39.257762 11371 ts_tablet_manager.cc:1428] T 685346c2a68e44cc8c1fbc3b8905c2b5 P 7205a2fa560f4b448715b67be73c1174: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 18:55:39.257848 11373 consensus_queue.cc:260] T 685346c2a68e44cc8c1fbc3b8905c2b5 P eb05f4d58b3f470881b5172e1f312e0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb05f4d58b3f470881b5172e1f312e0d" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38277 } } peers { permanent_uuid: "1c9d229dc7694099b581b5c31563108d" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 37907 } } peers { permanent_uuid: "7205a2fa560f4b448715b67be73c1174" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 43411 } }
I20260811 18:55:39.258035 11372 raft_consensus.cc:348] T 685346c2a68e44cc8c1fbc3b8905c2b5 P 1c9d229dc7694099b581b5c31563108d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb05f4d58b3f470881b5172e1f312e0d" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38277 } } peers { permanent_uuid: "1c9d229dc7694099b581b5c31563108d" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 37907 } } peers { permanent_uuid: "7205a2fa560f4b448715b67be73c1174" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 43411 } }
I20260811 18:55:39.258117 11373 ts_tablet_manager.cc:1428] T 685346c2a68e44cc8c1fbc3b8905c2b5 P eb05f4d58b3f470881b5172e1f312e0d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 18:55:39.258132 11372 raft_consensus.cc:374] T 685346c2a68e44cc8c1fbc3b8905c2b5 P 1c9d229dc7694099b581b5c31563108d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:55:39.258162 11372 raft_consensus.cc:729] T 685346c2a68e44cc8c1fbc3b8905c2b5 P 1c9d229dc7694099b581b5c31563108d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c9d229dc7694099b581b5c31563108d, State: Initialized, Role: FOLLOWER
I20260811 18:55:39.258270 11372 consensus_queue.cc:260] T 685346c2a68e44cc8c1fbc3b8905c2b5 P 1c9d229dc7694099b581b5c31563108d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb05f4d58b3f470881b5172e1f312e0d" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38277 } } peers { permanent_uuid: "1c9d229dc7694099b581b5c31563108d" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 37907 } } peers { permanent_uuid: "7205a2fa560f4b448715b67be73c1174" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 43411 } }
I20260811 18:55:39.258548 11372 ts_tablet_manager.cc:1428] T 685346c2a68e44cc8c1fbc3b8905c2b5 P 1c9d229dc7694099b581b5c31563108d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 18:55:39.415721 11386 raft_consensus.cc:482] T 685346c2a68e44cc8c1fbc3b8905c2b5 P 7205a2fa560f4b448715b67be73c1174 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:55:39.415839 11386 raft_consensus.cc:504] T 685346c2a68e44cc8c1fbc3b8905c2b5 P 7205a2fa560f4b448715b67be73c1174 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb05f4d58b3f470881b5172e1f312e0d" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38277 } } peers { permanent_uuid: "1c9d229dc7694099b581b5c31563108d" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 37907 } } peers { permanent_uuid: "7205a2fa560f4b448715b67be73c1174" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 43411 } }
I20260811 18:55:39.416226 11386 leader_election.cc:288] T 685346c2a68e44cc8c1fbc3b8905c2b5 P 7205a2fa560f4b448715b67be73c1174 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb05f4d58b3f470881b5172e1f312e0d (127.27.180.129:38277), 1c9d229dc7694099b581b5c31563108d (127.27.180.130:37907)
I20260811 18:55:39.416630 10995 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "685346c2a68e44cc8c1fbc3b8905c2b5" candidate_uuid: "7205a2fa560f4b448715b67be73c1174" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c9d229dc7694099b581b5c31563108d" is_pre_election: true
I20260811 18:55:39.416616 10758 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "685346c2a68e44cc8c1fbc3b8905c2b5" candidate_uuid: "7205a2fa560f4b448715b67be73c1174" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb05f4d58b3f470881b5172e1f312e0d" is_pre_election: true
I20260811 18:55:39.416779 10995 raft_consensus.cc:2454] T 685346c2a68e44cc8c1fbc3b8905c2b5 P 1c9d229dc7694099b581b5c31563108d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7205a2fa560f4b448715b67be73c1174 in term 0.
I20260811 18:55:39.416779 10758 raft_consensus.cc:2454] T 685346c2a68e44cc8c1fbc3b8905c2b5 P eb05f4d58b3f470881b5172e1f312e0d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7205a2fa560f4b448715b67be73c1174 in term 0.
I20260811 18:55:39.417148 11122 leader_election.cc:302] T 685346c2a68e44cc8c1fbc3b8905c2b5 P 7205a2fa560f4b448715b67be73c1174 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1c9d229dc7694099b581b5c31563108d, 7205a2fa560f4b448715b67be73c1174, eb05f4d58b3f470881b5172e1f312e0d; no voters:
I20260811 18:55:39.417338 11386 raft_consensus.cc:2781] T 685346c2a68e44cc8c1fbc3b8905c2b5 P 7205a2fa560f4b448715b67be73c1174 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:55:39.417413 11386 raft_consensus.cc:482] T 685346c2a68e44cc8c1fbc3b8905c2b5 P 7205a2fa560f4b448715b67be73c1174 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:55:39.417440 11386 raft_consensus.cc:3037] T 685346c2a68e44cc8c1fbc3b8905c2b5 P 7205a2fa560f4b448715b67be73c1174 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:55:39.418386 11386 raft_consensus.cc:504] T 685346c2a68e44cc8c1fbc3b8905c2b5 P 7205a2fa560f4b448715b67be73c1174 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb05f4d58b3f470881b5172e1f312e0d" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38277 } } peers { permanent_uuid: "1c9d229dc7694099b581b5c31563108d" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 37907 } } peers { permanent_uuid: "7205a2fa560f4b448715b67be73c1174" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 43411 } }
I20260811 18:55:39.418732 11386 leader_election.cc:288] T 685346c2a68e44cc8c1fbc3b8905c2b5 P 7205a2fa560f4b448715b67be73c1174 [CANDIDATE]: Term 1 election: Requested vote from peers eb05f4d58b3f470881b5172e1f312e0d (127.27.180.129:38277), 1c9d229dc7694099b581b5c31563108d (127.27.180.130:37907)
I20260811 18:55:39.419050 10758 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "685346c2a68e44cc8c1fbc3b8905c2b5" candidate_uuid: "7205a2fa560f4b448715b67be73c1174" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb05f4d58b3f470881b5172e1f312e0d"
I20260811 18:55:39.419096 10995 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "685346c2a68e44cc8c1fbc3b8905c2b5" candidate_uuid: "7205a2fa560f4b448715b67be73c1174" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c9d229dc7694099b581b5c31563108d"
I20260811 18:55:39.419183 10758 raft_consensus.cc:3037] T 685346c2a68e44cc8c1fbc3b8905c2b5 P eb05f4d58b3f470881b5172e1f312e0d [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:55:39.419202 10995 raft_consensus.cc:3037] T 685346c2a68e44cc8c1fbc3b8905c2b5 P 1c9d229dc7694099b581b5c31563108d [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:55:39.420109 10995 raft_consensus.cc:2454] T 685346c2a68e44cc8c1fbc3b8905c2b5 P 1c9d229dc7694099b581b5c31563108d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7205a2fa560f4b448715b67be73c1174 in term 1.
I20260811 18:55:39.420141 10758 raft_consensus.cc:2454] T 685346c2a68e44cc8c1fbc3b8905c2b5 P eb05f4d58b3f470881b5172e1f312e0d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7205a2fa560f4b448715b67be73c1174 in term 1.
I20260811 18:55:39.420368 11122 leader_election.cc:302] T 685346c2a68e44cc8c1fbc3b8905c2b5 P 7205a2fa560f4b448715b67be73c1174 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c9d229dc7694099b581b5c31563108d, 7205a2fa560f4b448715b67be73c1174; no voters:
I20260811 18:55:39.420621 11386 raft_consensus.cc:2781] T 685346c2a68e44cc8c1fbc3b8905c2b5 P 7205a2fa560f4b448715b67be73c1174 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:55:39.420785 11386 raft_consensus.cc:686] T 685346c2a68e44cc8c1fbc3b8905c2b5 P 7205a2fa560f4b448715b67be73c1174 [term 1 LEADER]: Becoming Leader. State: Replica: 7205a2fa560f4b448715b67be73c1174, State: Running, Role: LEADER
I20260811 18:55:39.420930 11386 consensus_queue.cc:237] T 685346c2a68e44cc8c1fbc3b8905c2b5 P 7205a2fa560f4b448715b67be73c1174 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb05f4d58b3f470881b5172e1f312e0d" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38277 } } peers { permanent_uuid: "1c9d229dc7694099b581b5c31563108d" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 37907 } } peers { permanent_uuid: "7205a2fa560f4b448715b67be73c1174" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 43411 } }
I20260811 18:55:39.422501 10460 catalog_manager.cc:5495] T 685346c2a68e44cc8c1fbc3b8905c2b5 P 7205a2fa560f4b448715b67be73c1174 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7205a2fa560f4b448715b67be73c1174 (127.27.180.131). New cstate: current_term: 1 leader_uuid: "7205a2fa560f4b448715b67be73c1174" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb05f4d58b3f470881b5172e1f312e0d" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c9d229dc7694099b581b5c31563108d" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 37907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7205a2fa560f4b448715b67be73c1174" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 43411 } health_report { overall_health: HEALTHY } } }
I20260811 18:55:39.455200 11367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:39.452737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37358 (local address 127.27.180.131:43411)
0811 18:55:39.452839 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:39.452842 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:39.452855 (+ 13us) server_negotiation.cc:407] Connection header received
0811 18:55:39.452941 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:39.452943 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:39.452983 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:39.453039 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:55:39.453253 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:39.453731 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:39.454435 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:39.454629 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:39.454692 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:39.454858 (+ 166us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:55:39.454928 (+ 70us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:55:39.454974 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:39.455033 (+ 59us) server_negotiation.cc:299] Negotiation successful
0811 18:55:39.455072 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260811 18:55:39.460698 10995 raft_consensus.cc:1261] T 685346c2a68e44cc8c1fbc3b8905c2b5 P 1c9d229dc7694099b581b5c31563108d [term 1 FOLLOWER]: Refusing update from remote peer 7205a2fa560f4b448715b67be73c1174: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:55:39.460708 10758 raft_consensus.cc:1261] T 685346c2a68e44cc8c1fbc3b8905c2b5 P eb05f4d58b3f470881b5172e1f312e0d [term 1 FOLLOWER]: Refusing update from remote peer 7205a2fa560f4b448715b67be73c1174: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:55:39.461126 11546 consensus_queue.cc:1035] T 685346c2a68e44cc8c1fbc3b8905c2b5 P 7205a2fa560f4b448715b67be73c1174 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c9d229dc7694099b581b5c31563108d" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 37907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:55:39.461277 11386 consensus_queue.cc:1035] T 685346c2a68e44cc8c1fbc3b8905c2b5 P 7205a2fa560f4b448715b67be73c1174 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb05f4d58b3f470881b5172e1f312e0d" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:55:39.490224 10460 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35638:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:55:39.490368 10460 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35638:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:55:39.492189 10460 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8627c387211147ed82345656961a4525: Sending DeleteTablet for 3 replicas of tablet 685346c2a68e44cc8c1fbc3b8905c2b5
I20260811 18:55:39.492686 10721 tablet_service.cc:1514] Processing DeleteTablet for tablet 685346c2a68e44cc8c1fbc3b8905c2b5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:55:39 UTC) from {username='oryx', principal='oryx/127.27.180.190@KRBTEST.COM'} at 127.0.0.1:38594
I20260811 18:55:39.492728 10958 tablet_service.cc:1514] Processing DeleteTablet for tablet 685346c2a68e44cc8c1fbc3b8905c2b5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:55:39 UTC) from {username='oryx', principal='oryx/127.27.180.190@KRBTEST.COM'} at 127.0.0.1:57478
I20260811 18:55:39.492812 11207 tablet_service.cc:1514] Processing DeleteTablet for tablet 685346c2a68e44cc8c1fbc3b8905c2b5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:55:39 UTC) from {username='oryx', principal='oryx/127.27.180.190@KRBTEST.COM'} at 127.0.0.1:37328
I20260811 18:55:39.492871 11421 tablet_replica.cc:330] T 685346c2a68e44cc8c1fbc3b8905c2b5 P eb05f4d58b3f470881b5172e1f312e0d: stopping tablet replica
I20260811 18:55:39.492964 11422 tablet_replica.cc:330] T 685346c2a68e44cc8c1fbc3b8905c2b5 P 1c9d229dc7694099b581b5c31563108d: stopping tablet replica
I20260811 18:55:39.492983 11424 tablet_replica.cc:330] T 685346c2a68e44cc8c1fbc3b8905c2b5 P 7205a2fa560f4b448715b67be73c1174: stopping tablet replica
I20260811 18:55:39.492985 11421 raft_consensus.cc:2229] T 685346c2a68e44cc8c1fbc3b8905c2b5 P eb05f4d58b3f470881b5172e1f312e0d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:55:39.493070 11424 raft_consensus.cc:2229] T 685346c2a68e44cc8c1fbc3b8905c2b5 P 7205a2fa560f4b448715b67be73c1174 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:55:39.493109 11422 raft_consensus.cc:2229] T 685346c2a68e44cc8c1fbc3b8905c2b5 P 1c9d229dc7694099b581b5c31563108d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:55:39.493126 11421 raft_consensus.cc:2258] T 685346c2a68e44cc8c1fbc3b8905c2b5 P eb05f4d58b3f470881b5172e1f312e0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:55:39.493247 11422 raft_consensus.cc:2258] T 685346c2a68e44cc8c1fbc3b8905c2b5 P 1c9d229dc7694099b581b5c31563108d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:55:39.493276 11424 raft_consensus.cc:2258] T 685346c2a68e44cc8c1fbc3b8905c2b5 P 7205a2fa560f4b448715b67be73c1174 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:55:39.493782 11421 ts_tablet_manager.cc:1905] T 685346c2a68e44cc8c1fbc3b8905c2b5 P eb05f4d58b3f470881b5172e1f312e0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:55:39.493907 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 10594
I20260811 18:55:39.494012 11422 ts_tablet_manager.cc:1905] T 685346c2a68e44cc8c1fbc3b8905c2b5 P 1c9d229dc7694099b581b5c31563108d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:55:39.495857 11424 ts_tablet_manager.cc:1905] T 685346c2a68e44cc8c1fbc3b8905c2b5 P 7205a2fa560f4b448715b67be73c1174: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:55:39.496522 11422 ts_tablet_manager.cc:1918] T 685346c2a68e44cc8c1fbc3b8905c2b5 P 1c9d229dc7694099b581b5c31563108d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 18:55:39.496595 11422 log.cc:1196] T 685346c2a68e44cc8c1fbc3b8905c2b5 P 1c9d229dc7694099b581b5c31563108d: Deleting WAL directory at /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/minicluster-data/ts-1/wal/wals/685346c2a68e44cc8c1fbc3b8905c2b5
I20260811 18:55:39.496850 11422 ts_tablet_manager.cc:1939] T 685346c2a68e44cc8c1fbc3b8905c2b5 P 1c9d229dc7694099b581b5c31563108d: Deleting consensus metadata
I20260811 18:55:39.497664 10436 catalog_manager.cc:4841] TS 1c9d229dc7694099b581b5c31563108d (127.27.180.130:37907): tablet 685346c2a68e44cc8c1fbc3b8905c2b5 (table test-table [id=ea384bf831174be5bcf41124bb5b1f35]) successfully deleted
I20260811 18:55:39.498003 11424 ts_tablet_manager.cc:1918] T 685346c2a68e44cc8c1fbc3b8905c2b5 P 7205a2fa560f4b448715b67be73c1174: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 18:55:39.498052 11424 log.cc:1196] T 685346c2a68e44cc8c1fbc3b8905c2b5 P 7205a2fa560f4b448715b67be73c1174: Deleting WAL directory at /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786474496157106-28370-0/minicluster-data/ts-2/wal/wals/685346c2a68e44cc8c1fbc3b8905c2b5
I20260811 18:55:39.498210 11424 ts_tablet_manager.cc:1939] T 685346c2a68e44cc8c1fbc3b8905c2b5 P 7205a2fa560f4b448715b67be73c1174: Deleting consensus metadata
I20260811 18:55:39.498871 10430 catalog_manager.cc:4841] TS 7205a2fa560f4b448715b67be73c1174 (127.27.180.131:43411): tablet 685346c2a68e44cc8c1fbc3b8905c2b5 (table test-table [id=ea384bf831174be5bcf41124bb5b1f35]) successfully deleted
W20260811 18:55:39.505134 10430 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.27.180.129:38277 (error 108)
I20260811 18:55:39.505942 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 10845
I20260811 18:55:39.506160 11553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:39.505777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.129:38277 (local address 127.0.0.1:38610)
0811 18:55:39.505989 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:39.506066 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.129:38277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":57,"threads_started":1}
W20260811 18:55:39.514564 10430 catalog_manager.cc:4584] TS eb05f4d58b3f470881b5172e1f312e0d (127.27.180.129:38277): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 685346c2a68e44cc8c1fbc3b8905c2b5: Network error: Client connection negotiation failed: client connection to 127.27.180.129:38277: connect: Connection refused (error 111)
I20260811 18:55:39.522354 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 11091
I20260811 18:55:39.541100 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 10401
2026-08-11T18:55:39Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3799 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786474496157106-28370-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:55:39 dist-test-slave-2j7r krb5kdc[11562](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:55:39 dist-test-slave-2j7r krb5kdc[11562](info): set up 2 sockets
Aug 11 18:55:39 dist-test-slave-2j7r krb5kdc[11562](info): commencing operation
krb5kdc: starting...
W20260811 18:55:40.732296 28370 mini_kdc.cc:121] Time spent starting KDC: real 1.159s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786474496157106-28370-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786474496157106-28370-0/krb5kdc/test-user.keytab.
Aug 11 18:55:40 dist-test-slave-2j7r krb5kdc[11562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474540, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:55:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:55:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.180.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.180.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.180.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab.
Entry for principal kudu/127.27.180.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab.
I20260811 18:55:40.849617 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786474496157106-28370-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786474496157106-28370-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786474496157106-28370-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.190:42225
--webserver_interface=127.27.180.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab
--principal=kudu/127.27.180.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:33451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.180.190:42225
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:55:40.959296 11580 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:55:40.959581 11580 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:55:40.959653 11580 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:55:40.963572 11580 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:55:40.963624 11580 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:55:40.963651 11580 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:55:40.963673 11580 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:55:40.963701 11580 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:55:40.968719 11580 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:33451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786474496157106-28370-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786474496157106-28370-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.180.190:42225
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.190:42225
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.180.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11580
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786474496157106-28370-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:55:40.969681 11580 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:55:40.970620 11580 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:55:40.978698 11592 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:55:40.978855 11580 server_base.cc:1034] running on GCE node
W20260811 18:55:40.978700 11587 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:40.978700 11594 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:55:40.979571 11580 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:55:40.980597 11580 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:55:40.981760 11580 hybrid_clock.cc:648] HybridClock initialized: now 1786474540981723 us; error 38 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-11T18:55:41Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1445 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:55:41 dist-test-slave-2j7r krb5kdc[11605](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:55:41 dist-test-slave-2j7r krb5kdc[11605](info): set up 2 sockets
Aug 11 18:55:41 dist-test-slave-2j7r krb5kdc[11605](info): commencing operation
krb5kdc: starting...
W20260811 18:55:42.175233 28370 mini_kdc.cc:121] Time spent starting KDC: real 1.157s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/krb5kdc/test-user.keytab.
Aug 11 18:55:42 dist-test-slave-2j7r krb5kdc[11605](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474542, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:55:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:55:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.180.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.180.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.180.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab.
Entry for principal kudu/127.27.180.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab.
I20260811 18:55:42.292140 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.190:33277
--webserver_interface=127.27.180.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab
--principal=kudu/127.27.180.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:35755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.180.190:33277
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:55:42.401032 11630 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:55:42.401288 11630 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:55:42.401343 11630 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:55:42.405237 11630 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:55:42.405290 11630 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:55:42.405314 11630 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:55:42.405335 11630 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:55:42.405360 11630 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:55:42.410624 11630 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:35755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.180.190:33277
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab
--principal=kudu/127.27.180.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.190:33277
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.180.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11630
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:55:42.411613 11630 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:55:42.412590 11630 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:55:42.420758 11637 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:42.420825 11644 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:42.420778 11638 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:55:42.421048 11630 server_base.cc:1034] running on GCE node
I20260811 18:55:42.421681 11630 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:55:42.422747 11630 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:55:42.423906 11630 hybrid_clock.cc:648] HybridClock initialized: now 1786474542423886 us; error 32 us; skew 500 ppm
Aug 11 18:55:42 dist-test-slave-2j7r krb5kdc[11605](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474542, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:55:42.426604 11630 init.cc:377] Logged in from keytab as kudu/127.27.180.190@KRBTEST.COM (short username kudu)
I20260811 18:55:42.427524 11630 webserver.cc:460] Webserver started at http://127.27.180.190:45065/ using document root <none> and password file <none>
I20260811 18:55:42.428063 11630 fs_manager.cc:359] Metadata directory not provided
I20260811 18:55:42.428130 11630 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:55:42.428344 11630 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:55:42.430413 11630 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/master-0/data/instance:
uuid: "bc353283acca46a9940dad138c2cc368"
format_stamp: "Formatted at 2026-08-11 18:55:42 on dist-test-slave-2j7r"
server_key: "967ada22da4018fa3c3abcc0e5f31df0"
server_key_iv: "65aba10f56c8fc90df516cd3bd393ac2"
server_key_version: "encryptionkey@0"
I20260811 18:55:42.430951 11630 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/master-0/wal/instance:
uuid: "bc353283acca46a9940dad138c2cc368"
format_stamp: "Formatted at 2026-08-11 18:55:42 on dist-test-slave-2j7r"
server_key: "967ada22da4018fa3c3abcc0e5f31df0"
server_key_iv: "65aba10f56c8fc90df516cd3bd393ac2"
server_key_version: "encryptionkey@0"
I20260811 18:55:42.434522 11630 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 18:55:42.436721 11661 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:42.437907 11630 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 18:55:42.438031 11630 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/master-0/wal
uuid: "bc353283acca46a9940dad138c2cc368"
format_stamp: "Formatted at 2026-08-11 18:55:42 on dist-test-slave-2j7r"
server_key: "967ada22da4018fa3c3abcc0e5f31df0"
server_key_iv: "65aba10f56c8fc90df516cd3bd393ac2"
server_key_version: "encryptionkey@0"
I20260811 18:55:42.438120 11630 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:55:42.451388 11630 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:55:42.453926 11630 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:55:42.454092 11630 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:55:42.460778 11630 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.190:33277
I20260811 18:55:42.460817 11765 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.190:33277 every 8 connection(s)
I20260811 18:55:42.461721 11630 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
I20260811 18:55:42.464362 11768 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:55:42.467779 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 11630
I20260811 18:55:42.467932 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/master-0/wal/instance
I20260811 18:55:42.468271 28370 external_mini_cluster.cc:1428] Setting key bc50f008f06a32d0161096eacfd937da
I20260811 18:55:42.470010 11768 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc353283acca46a9940dad138c2cc368: Bootstrap starting.
I20260811 18:55:42.472342 11768 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bc353283acca46a9940dad138c2cc368: Neither blocks nor log segments found. Creating new log.
I20260811 18:55:42.473055 11768 log.cc:824] T 00000000000000000000000000000000 P bc353283acca46a9940dad138c2cc368: Log is configured to *not* fsync() on all Append() calls
I20260811 18:55:42.475102 11768 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc353283acca46a9940dad138c2cc368: No bootstrap required, opened a new log
Aug 11 18:55:42 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474542, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM
I20260811 18:55:42.478921 11768 raft_consensus.cc:348] T 00000000000000000000000000000000 P bc353283acca46a9940dad138c2cc368 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc353283acca46a9940dad138c2cc368" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 33277 } }
I20260811 18:55:42.479128 11768 raft_consensus.cc:374] T 00000000000000000000000000000000 P bc353283acca46a9940dad138c2cc368 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:55:42.479179 11768 raft_consensus.cc:729] T 00000000000000000000000000000000 P bc353283acca46a9940dad138c2cc368 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc353283acca46a9940dad138c2cc368, State: Initialized, Role: FOLLOWER
I20260811 18:55:42.479605 11768 consensus_queue.cc:260] T 00000000000000000000000000000000 P bc353283acca46a9940dad138c2cc368 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc353283acca46a9940dad138c2cc368" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 33277 } }
I20260811 18:55:42.479740 11768 raft_consensus.cc:388] T 00000000000000000000000000000000 P bc353283acca46a9940dad138c2cc368 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:55:42.479800 11768 raft_consensus.cc:482] T 00000000000000000000000000000000 P bc353283acca46a9940dad138c2cc368 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:55:42.479890 11768 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bc353283acca46a9940dad138c2cc368 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:55:42.480715 11768 raft_consensus.cc:504] T 00000000000000000000000000000000 P bc353283acca46a9940dad138c2cc368 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc353283acca46a9940dad138c2cc368" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 33277 } }
I20260811 18:55:42.481024 11768 leader_election.cc:302] T 00000000000000000000000000000000 P bc353283acca46a9940dad138c2cc368 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bc353283acca46a9940dad138c2cc368; no voters:
I20260811 18:55:42.481319 11768 leader_election.cc:288] T 00000000000000000000000000000000 P bc353283acca46a9940dad138c2cc368 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:55:42.481426 11781 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bc353283acca46a9940dad138c2cc368 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:55:42.481582 11781 raft_consensus.cc:686] T 00000000000000000000000000000000 P bc353283acca46a9940dad138c2cc368 [term 1 LEADER]: Becoming Leader. State: Replica: bc353283acca46a9940dad138c2cc368, State: Running, Role: LEADER
I20260811 18:55:42.481860 11781 consensus_queue.cc:237] T 00000000000000000000000000000000 P bc353283acca46a9940dad138c2cc368 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc353283acca46a9940dad138c2cc368" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 33277 } }
I20260811 18:55:42.482101 11773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:42.469612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33674 (local address 127.27.180.190:33277)
0811 18:55:42.469995 (+ 383us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:42.470010 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:42.470046 (+ 36us) server_negotiation.cc:407] Connection header received
0811 18:55:42.470684 (+ 638us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:42.470700 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:42.471014 (+ 314us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:42.471271 (+ 257us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:42.472114 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:42.473159 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:42.474119 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:42.474568 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:42.474682 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:42.477334 (+ 2652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:42.477339 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:42.477352 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:42.477379 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:42.478923 (+ 1544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:42.479390 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:42.479395 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:42.479399 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:42.479516 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:42.479805 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:42.479808 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:42.479810 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:42.480251 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:42.480385 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:42.480796 (+ 411us) server_negotiation.cc:299] Negotiation successful
0811 18:55:42.481137 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":255,"thread_start_us":103,"threads_started":1}
I20260811 18:55:42.482578 11768 sys_catalog.cc:564] T 00000000000000000000000000000000 P bc353283acca46a9940dad138c2cc368 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:55:42.483016 11781 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc353283acca46a9940dad138c2cc368 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bc353283acca46a9940dad138c2cc368. Latest consensus state: current_term: 1 leader_uuid: "bc353283acca46a9940dad138c2cc368" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc353283acca46a9940dad138c2cc368" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 33277 } } }
I20260811 18:55:42.483124 11781 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc353283acca46a9940dad138c2cc368 [sys.catalog]: This master's current role is: LEADER
I20260811 18:55:42.483399 11782 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc353283acca46a9940dad138c2cc368 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bc353283acca46a9940dad138c2cc368" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc353283acca46a9940dad138c2cc368" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 33277 } } }
I20260811 18:55:42.483489 11782 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc353283acca46a9940dad138c2cc368 [sys.catalog]: This master's current role is: LEADER
W20260811 18:55:42.486824 11808 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bc353283acca46a9940dad138c2cc368: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:55:42.486970 11808 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:55:42.487092 11787 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:55:42.487802 11787 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:55:42.492519 11787 catalog_manager.cc:1347] Generated new cluster ID: 7294f82406c7403a8adbf251def556b9
I20260811 18:55:42.492580 11787 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:55:42.503937 11787 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:55:42.505128 11787 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:55:42.510334 11787 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bc353283acca46a9940dad138c2cc368: Generated new TSK 0
I20260811 18:55:42.511024 11787 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.180.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.180.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.180.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.129.keytab.
Entry for principal kudu/127.27.180.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.129.keytab.
I20260811 18:55:42.544068 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.180.129:0
--local_ip_for_outbound_sockets=127.27.180.129
--webserver_interface=127.27.180.129
--webserver_port=0
--tserver_master_addrs=127.27.180.190:33277
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.129.keytab
--principal=kudu/127.27.180.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:35755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:55:42.654650 11817 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:55:42.654876 11817 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:55:42.654915 11817 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:55:42.658757 11817 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:55:42.658802 11817 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:55:42.658852 11817 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.180.129
I20260811 18:55:42.663823 11817 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:35755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.129.keytab
--principal=kudu/127.27.180.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.180.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.180.129
--webserver_port=0
--tserver_master_addrs=127.27.180.190:33277
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11817
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.180.129
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:55:42.664673 11817 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:55:42.665602 11817 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 18:55:42.674669 11817 server_base.cc:1034] running on GCE node
W20260811 18:55:42.674595 11822 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:42.674575 11828 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:42.674564 11824 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:55:42.675215 11817 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:55:42.675779 11817 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:55:42.676954 11817 hybrid_clock.cc:648] HybridClock initialized: now 1786474542676935 us; error 28 us; skew 500 ppm
Aug 11 18:55:42 dist-test-slave-2j7r krb5kdc[11605](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474542, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:55:42.681998 11817 init.cc:377] Logged in from keytab as kudu/127.27.180.129@KRBTEST.COM (short username kudu)
I20260811 18:55:42.682788 11817 webserver.cc:460] Webserver started at http://127.27.180.129:40035/ using document root <none> and password file <none>
I20260811 18:55:42.683295 11817 fs_manager.cc:359] Metadata directory not provided
I20260811 18:55:42.683351 11817 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:55:42.683532 11817 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:55:42.685519 11817 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/ts-0/data/instance:
uuid: "fbd4829ec716495e937f131fa96196a9"
format_stamp: "Formatted at 2026-08-11 18:55:42 on dist-test-slave-2j7r"
server_key: "5dd9e92532e63fe44b288dd95401939c"
server_key_iv: "c93016499851fd6de2e5a46c8cb1f333"
server_key_version: "encryptionkey@0"
I20260811 18:55:42.686036 11817 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/ts-0/wal/instance:
uuid: "fbd4829ec716495e937f131fa96196a9"
format_stamp: "Formatted at 2026-08-11 18:55:42 on dist-test-slave-2j7r"
server_key: "5dd9e92532e63fe44b288dd95401939c"
server_key_iv: "c93016499851fd6de2e5a46c8cb1f333"
server_key_version: "encryptionkey@0"
I20260811 18:55:42.689471 11817 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 18:55:42.691708 11846 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:42.692826 11817 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 18:55:42.692930 11817 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/ts-0/wal
uuid: "fbd4829ec716495e937f131fa96196a9"
format_stamp: "Formatted at 2026-08-11 18:55:42 on dist-test-slave-2j7r"
server_key: "5dd9e92532e63fe44b288dd95401939c"
server_key_iv: "c93016499851fd6de2e5a46c8cb1f333"
server_key_version: "encryptionkey@0"
I20260811 18:55:42.693020 11817 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:55:42.719833 11817 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:55:42.722812 11817 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:55:42.723001 11817 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:55:42.723609 11817 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:55:42.724478 11817 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:55:42.724534 11817 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:42.724594 11817 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:55:42.724622 11817 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:42.733697 11817 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.129:45869
I20260811 18:55:42.733718 12060 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.129:45869 every 8 connection(s)
I20260811 18:55:42.734570 11817 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/ts-0/data/info.pb
I20260811 18:55:42.740871 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 11817
I20260811 18:55:42.740988 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/ts-0/wal/instance
I20260811 18:55:42.741290 28370 external_mini_cluster.cc:1428] Setting key 77f3c30f18cc15ce6102a7f37e2bb9b6
Aug 11 18:55:42 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474542, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.129@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM
I20260811 18:55:42.747898 11773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:42.736317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.129:54125 (local address 127.27.180.190:33277)
0811 18:55:42.736464 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:42.736469 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:42.737343 (+ 874us) server_negotiation.cc:407] Connection header received
0811 18:55:42.738587 (+ 1244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:42.738590 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:42.738642 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:42.738741 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:42.739833 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:42.740375 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:42.741710 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:42.741982 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:42.742067 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:42.744590 (+ 2523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:42.744595 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:42.744597 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:42.744625 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:42.745906 (+ 1281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:42.746437 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:42.746439 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:42.746440 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:42.746503 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:42.746822 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:42.746824 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:42.746825 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:42.746998 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:42.747067 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:42.747647 (+ 580us) server_negotiation.cc:299] Negotiation successful
0811 18:55:42.747769 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 18:55:42.748968 12065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:42.736743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33277 (local address 127.27.180.129:54125)
0811 18:55:42.737205 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:42.737247 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:42.738376 (+ 1129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:42.738886 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:42.738892 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:42.739226 (+ 334us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:42.739687 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:42.739701 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:42.740473 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:42.740476 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:42.741590 (+ 1114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:42.741603 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:42.742131 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:42.742135 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:42.742331 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:42.742915 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:42.742932 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:42.744428 (+ 1496us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:55:42.746028 (+ 1600us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:42.746033 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:42.746045 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:42.746352 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:42.746574 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:42.746577 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:42.746578 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:42.746747 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:42.747087 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:55:42.747093 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:55:42.747406 (+ 313us) client_negotiation.cc:769] Sending connection context
0811 18:55:42.747626 (+ 220us) client_negotiation.cc:240] Negotiation successful
0811 18:55:42.747963 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":121,"threads_started":1}
I20260811 18:55:42.750084 12063 heartbeater.cc:347] Connected to a master server at 127.27.180.190:33277
I20260811 18:55:42.750286 12063 heartbeater.cc:464] Registering TS with master...
I20260811 18:55:42.750725 12063 heartbeater.cc:511] Master 127.27.180.190:33277 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.27.180.130@KRBTEST.COM; defaulting to no policy
I20260811 18:55:42.752208 11692 ts_manager.cc:194] Registered new tserver with Master: fbd4829ec716495e937f131fa96196a9 (127.27.180.129:45869)
I20260811 18:55:42.753324 11692 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.180.129@KRBTEST.COM'} at 127.27.180.129:54125
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.180.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.180.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.130.keytab.
Entry for principal kudu/127.27.180.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.130.keytab.
I20260811 18:55:42.767617 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.180.130:0
--local_ip_for_outbound_sockets=127.27.180.130
--webserver_interface=127.27.180.130
--webserver_port=0
--tserver_master_addrs=127.27.180.190:33277
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.130.keytab
--principal=kudu/127.27.180.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:35755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:55:42.886143 12069 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:55:42.886417 12069 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:55:42.886462 12069 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:55:42.890372 12069 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:55:42.890424 12069 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:55:42.890477 12069 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.180.130
I20260811 18:55:42.895457 12069 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:35755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.130.keytab
--principal=kudu/127.27.180.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.180.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.180.130
--webserver_port=0
--tserver_master_addrs=127.27.180.190:33277
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12069
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.180.130
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:55:42.896349 12069 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:55:42.897334 12069 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:55:42.906499 12079 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:42.906517 12084 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:55:42.906574 12069 server_base.cc:1034] running on GCE node
W20260811 18:55:42.906561 12080 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:55:42.907115 12069 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:55:42.907701 12069 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:55:42.908854 12069 hybrid_clock.cc:648] HybridClock initialized: now 1786474542908834 us; error 31 us; skew 500 ppm
Aug 11 18:55:42 dist-test-slave-2j7r krb5kdc[11605](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474542, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:55:42.911437 12069 init.cc:377] Logged in from keytab as kudu/127.27.180.130@KRBTEST.COM (short username kudu)
I20260811 18:55:42.912245 12069 webserver.cc:460] Webserver started at http://127.27.180.130:36351/ using document root <none> and password file <none>
I20260811 18:55:42.912751 12069 fs_manager.cc:359] Metadata directory not provided
I20260811 18:55:42.912810 12069 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:55:42.913012 12069 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:55:42.915031 12069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/ts-1/data/instance:
uuid: "a00e8c38d9e049089b8a735997cf8777"
format_stamp: "Formatted at 2026-08-11 18:55:42 on dist-test-slave-2j7r"
server_key: "c5917a56e31bc1cd9826b818cbeed678"
server_key_iv: "592ff44ecfdd32db2147bcf17618803e"
server_key_version: "encryptionkey@0"
I20260811 18:55:42.915565 12069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/ts-1/wal/instance:
uuid: "a00e8c38d9e049089b8a735997cf8777"
format_stamp: "Formatted at 2026-08-11 18:55:42 on dist-test-slave-2j7r"
server_key: "c5917a56e31bc1cd9826b818cbeed678"
server_key_iv: "592ff44ecfdd32db2147bcf17618803e"
server_key_version: "encryptionkey@0"
I20260811 18:55:42.919054 12069 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 18:55:42.921401 12103 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:42.922477 12069 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 18:55:42.922575 12069 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/ts-1/wal
uuid: "a00e8c38d9e049089b8a735997cf8777"
format_stamp: "Formatted at 2026-08-11 18:55:42 on dist-test-slave-2j7r"
server_key: "c5917a56e31bc1cd9826b818cbeed678"
server_key_iv: "592ff44ecfdd32db2147bcf17618803e"
server_key_version: "encryptionkey@0"
I20260811 18:55:42.922667 12069 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:55:42.938058 12069 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:55:42.940867 12069 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:55:42.941025 12069 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:55:42.941612 12069 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:55:42.942379 12069 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:55:42.942435 12069 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:42.942472 12069 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:55:42.942498 12069 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:42.950827 12069 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.130:39039
I20260811 18:55:42.950852 12329 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.130:39039 every 8 connection(s)
I20260811 18:55:42.951645 12069 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/ts-1/data/info.pb
I20260811 18:55:42.955603 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 12069
I20260811 18:55:42.955749 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/ts-1/wal/instance
I20260811 18:55:42.956096 28370 external_mini_cluster.cc:1428] Setting key efbb507cc931ebe7b20c9232e1c4fc52
Aug 11 18:55:42 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474542, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.130@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM
I20260811 18:55:42.964941 11773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:42.953301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.130:43369 (local address 127.27.180.190:33277)
0811 18:55:42.953431 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:42.953435 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:42.954233 (+ 798us) server_negotiation.cc:407] Connection header received
0811 18:55:42.955495 (+ 1262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:42.955498 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:42.955544 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:42.955614 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:42.956899 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:42.957637 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:42.959092 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:42.959307 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:42.959379 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:42.961691 (+ 2312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:42.961694 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:42.961696 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:42.961719 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:42.962983 (+ 1264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:42.963483 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:42.963487 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:42.963489 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:42.963586 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:42.963873 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:42.963884 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:42.963886 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:42.964123 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:42.964229 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:42.964642 (+ 413us) server_negotiation.cc:299] Negotiation successful
0811 18:55:42.964768 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 18:55:42.965633 12341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:42.953668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33277 (local address 127.27.180.130:43369)
0811 18:55:42.954101 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:42.954139 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:42.955280 (+ 1141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:42.955818 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:42.955826 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:42.956286 (+ 460us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:42.956792 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:42.956806 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:42.957762 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:42.957765 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:42.958988 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:42.959000 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:42.959436 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:42.959440 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:42.959630 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:42.960163 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:42.960178 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:42.961538 (+ 1360us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:55:42.963111 (+ 1573us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:42.963117 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:42.963130 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:42.963380 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:42.963671 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:42.963673 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:42.963674 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:42.963792 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:42.964212 (+ 420us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:55:42.964217 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:55:42.964424 (+ 207us) client_negotiation.cc:769] Sending connection context
0811 18:55:42.964569 (+ 145us) client_negotiation.cc:240] Negotiation successful
0811 18:55:42.964795 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":122,"threads_started":1}
WARNING: no policy specified for kudu/127.27.180.131@KRBTEST.COM; defaulting to no policy
I20260811 18:55:42.966688 12330 heartbeater.cc:347] Connected to a master server at 127.27.180.190:33277
I20260811 18:55:42.966899 12330 heartbeater.cc:464] Registering TS with master...
I20260811 18:55:42.967365 12330 heartbeater.cc:511] Master 127.27.180.190:33277 requested a full tablet report, sending...
I20260811 18:55:42.968312 11692 ts_manager.cc:194] Registered new tserver with Master: a00e8c38d9e049089b8a735997cf8777 (127.27.180.130:39039)
I20260811 18:55:42.968874 11692 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.180.130@KRBTEST.COM'} at 127.27.180.130:43369
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.180.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.180.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.131.keytab.
Entry for principal kudu/127.27.180.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.131.keytab.
I20260811 18:55:42.981576 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.180.131:0
--local_ip_for_outbound_sockets=127.27.180.131
--webserver_interface=127.27.180.131
--webserver_port=0
--tserver_master_addrs=127.27.180.190:33277
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.131.keytab
--principal=kudu/127.27.180.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:35755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:55:43.089246 12348 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:55:43.089476 12348 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:55:43.089522 12348 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:55:43.093446 12348 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:55:43.093499 12348 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:55:43.093559 12348 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.180.131
I20260811 18:55:43.098681 12348 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:35755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.131.keytab
--principal=kudu/127.27.180.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.180.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.180.131
--webserver_port=0
--tserver_master_addrs=127.27.180.190:33277
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12348
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.180.131
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:55:43.099647 12348 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:55:43.100621 12348 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:55:43.109884 12361 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:43.109872 12359 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:43.109871 12358 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:55:43.110137 12348 server_base.cc:1034] running on GCE node
I20260811 18:55:43.110525 12348 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:55:43.111168 12348 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:55:43.112339 12348 hybrid_clock.cc:648] HybridClock initialized: now 1786474543112321 us; error 34 us; skew 500 ppm
Aug 11 18:55:43 dist-test-slave-2j7r krb5kdc[11605](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474543, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:55:43.114910 12348 init.cc:377] Logged in from keytab as kudu/127.27.180.131@KRBTEST.COM (short username kudu)
I20260811 18:55:43.115741 12348 webserver.cc:460] Webserver started at http://127.27.180.131:37449/ using document root <none> and password file <none>
I20260811 18:55:43.116256 12348 fs_manager.cc:359] Metadata directory not provided
I20260811 18:55:43.116330 12348 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:55:43.116524 12348 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:55:43.118543 12348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/ts-2/data/instance:
uuid: "16496d53dfae4c21bf26cd760dd9487f"
format_stamp: "Formatted at 2026-08-11 18:55:43 on dist-test-slave-2j7r"
server_key: "e26cb700a92907f9f88352968e6a78a9"
server_key_iv: "6dd4b84ea6a7295ec42eaafa908769c9"
server_key_version: "encryptionkey@0"
I20260811 18:55:43.119077 12348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/ts-2/wal/instance:
uuid: "16496d53dfae4c21bf26cd760dd9487f"
format_stamp: "Formatted at 2026-08-11 18:55:43 on dist-test-slave-2j7r"
server_key: "e26cb700a92907f9f88352968e6a78a9"
server_key_iv: "6dd4b84ea6a7295ec42eaafa908769c9"
server_key_version: "encryptionkey@0"
I20260811 18:55:43.122586 12348 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.004s
I20260811 18:55:43.124850 12376 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:43.126029 12348 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 18:55:43.126133 12348 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/ts-2/wal
uuid: "16496d53dfae4c21bf26cd760dd9487f"
format_stamp: "Formatted at 2026-08-11 18:55:43 on dist-test-slave-2j7r"
server_key: "e26cb700a92907f9f88352968e6a78a9"
server_key_iv: "6dd4b84ea6a7295ec42eaafa908769c9"
server_key_version: "encryptionkey@0"
I20260811 18:55:43.126255 12348 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:55:43.163357 12348 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:55:43.166328 12348 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:55:43.166505 12348 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:55:43.167097 12348 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:55:43.167935 12348 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:55:43.167980 12348 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:43.168025 12348 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:55:43.168053 12348 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:43.176769 12348 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.131:36715
I20260811 18:55:43.176810 12599 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.131:36715 every 8 connection(s)
I20260811 18:55:43.177650 12348 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/ts-2/data/info.pb
Aug 11 18:55:43 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474543, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.131@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM
I20260811 18:55:43.187372 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 12348
I20260811 18:55:43.187500 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/ts-2/wal/instance
I20260811 18:55:43.187801 28370 external_mini_cluster.cc:1428] Setting key c8469d2a83032dd3d2a978bca4405283
I20260811 18:55:43.190569 11773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:43.179355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.131:56909 (local address 127.27.180.190:33277)
0811 18:55:43.179487 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:43.179491 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:43.180272 (+ 781us) server_negotiation.cc:407] Connection header received
0811 18:55:43.181603 (+ 1331us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:43.181606 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:43.181669 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:43.181739 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:43.182904 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:43.183437 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:43.184638 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:43.184845 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:43.184927 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:43.187172 (+ 2245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:43.187176 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:43.187177 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:43.187202 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:43.188682 (+ 1480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:43.189220 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:43.189223 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:43.189225 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:43.189314 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:43.189644 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:43.189648 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:43.189649 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:43.189840 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:43.189944 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:43.190360 (+ 416us) server_negotiation.cc:299] Negotiation successful
0811 18:55:43.190455 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 18:55:43.191473 12603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:43.179691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33277 (local address 127.27.180.131:56909)
0811 18:55:43.180125 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:43.180165 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:43.181395 (+ 1230us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:43.181928 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:43.181935 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:43.182273 (+ 338us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:43.182730 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:43.182744 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:43.183578 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:43.183581 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:43.184509 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:43.184543 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:43.184970 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:43.184972 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:43.185142 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:43.185655 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:43.185671 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:43.187028 (+ 1357us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:55:43.188785 (+ 1757us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:43.188790 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:43.188804 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:43.189126 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:43.189395 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:55:43.189398 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:55:43.189400 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:55:43.189545 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:55:43.189954 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:55:43.189959 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:55:43.190170 (+ 211us) client_negotiation.cc:769] Sending connection context
0811 18:55:43.190314 (+ 144us) client_negotiation.cc:240] Negotiation successful
0811 18:55:43.190549 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":118,"threads_started":1}
I20260811 18:55:43.192468 12600 heartbeater.cc:347] Connected to a master server at 127.27.180.190:33277
I20260811 18:55:43.192682 12600 heartbeater.cc:464] Registering TS with master...
I20260811 18:55:43.193183 12600 heartbeater.cc:511] Master 127.27.180.190:33277 requested a full tablet report, sending...
I20260811 18:55:43.194083 11692 ts_manager.cc:194] Registered new tserver with Master: 16496d53dfae4c21bf26cd760dd9487f (127.27.180.131:36715)
I20260811 18:55:43.194597 11692 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.180.131@KRBTEST.COM'} at 127.27.180.131:56909
I20260811 18:55:43.201408 28370 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.27.180.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.27.180.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.27.180.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/krb5kdc/oryx_127.27.180.129.keytab.
Entry for principal oryx/127.27.180.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/krb5kdc/oryx_127.27.180.129.keytab.
WARNING: no policy specified for oryx/127.27.180.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.27.180.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.27.180.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/krb5kdc/oryx_127.27.180.130.keytab.
Entry for principal oryx/127.27.180.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/krb5kdc/oryx_127.27.180.130.keytab.
WARNING: no policy specified for oryx/127.27.180.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.27.180.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.27.180.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/krb5kdc/oryx_127.27.180.131.keytab.
Entry for principal oryx/127.27.180.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/krb5kdc/oryx_127.27.180.131.keytab.
I20260811 18:55:43.265364 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 11817
I20260811 18:55:43.278119 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 12069
I20260811 18:55:43.288879 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 12348
I20260811 18:55:43.335269 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 11630
I20260811 18:55:43.347296 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.190:33277
--webserver_interface=127.27.180.190
--webserver_port=45065
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab
--principal=kudu/127.27.180.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:35755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.180.190:33277
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:55:43.454682 12613 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:55:43.454944 12613 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:55:43.454999 12613 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:55:43.458895 12613 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:55:43.458943 12613 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:55:43.458968 12613 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:55:43.458988 12613 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:55:43.459010 12613 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:55:43.464129 12613 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:35755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.180.190:33277
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab
--principal=kudu/127.27.180.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.190:33277
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.180.190
--webserver_port=45065
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12613
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:55:43.465135 12613 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:55:43.466140 12613 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:55:43.474192 12630 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:43.474179 12625 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:43.474177 12626 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:55:43.474335 12613 server_base.cc:1034] running on GCE node
I20260811 18:55:43.474870 12613 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:55:43.475927 12613 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:55:43.477108 12613 hybrid_clock.cc:648] HybridClock initialized: now 1786474543477070 us; error 30 us; skew 500 ppm
Aug 11 18:55:43 dist-test-slave-2j7r krb5kdc[11605](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474543, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:55:43.479691 12613 init.cc:377] Logged in from keytab as kudu/127.27.180.190@KRBTEST.COM (short username kudu)
I20260811 18:55:43.480532 12613 webserver.cc:460] Webserver started at http://127.27.180.190:45065/ using document root <none> and password file <none>
I20260811 18:55:43.481048 12613 fs_manager.cc:359] Metadata directory not provided
I20260811 18:55:43.481153 12613 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:55:43.484751 12613 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.002s
I20260811 18:55:43.486785 12642 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:43.487987 12613 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260811 18:55:43.488088 12613 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/master-0/wal
uuid: "bc353283acca46a9940dad138c2cc368"
format_stamp: "Formatted at 2026-08-11 18:55:42 on dist-test-slave-2j7r"
server_key: "967ada22da4018fa3c3abcc0e5f31df0"
server_key_iv: "65aba10f56c8fc90df516cd3bd393ac2"
server_key_version: "encryptionkey@0"
I20260811 18:55:43.488600 12613 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:55:43.508456 12613 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:55:43.511121 12613 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:55:43.511302 12613 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:55:43.518144 12613 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.190:33277
I20260811 18:55:43.518162 12743 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.190:33277 every 8 connection(s)
I20260811 18:55:43.519146 12613 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
I20260811 18:55:43.522949 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 12613
I20260811 18:55:43.523415 12745 sys_catalog.cc:263] Verifying existing consensus state
I20260811 18:55:43.523454 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.180.129:45869
--local_ip_for_outbound_sockets=127.27.180.129
--tserver_master_addrs=127.27.180.190:33277
--webserver_port=40035
--webserver_interface=127.27.180.129
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.129.keytab
--principal=kudu/127.27.180.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:35755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.27.180.129
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/krb5kdc/oryx_127.27.180.129.keytab with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 18:55:43.525482 12745 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc353283acca46a9940dad138c2cc368: Bootstrap starting.
I20260811 18:55:43.542789 12745 log.cc:824] T 00000000000000000000000000000000 P bc353283acca46a9940dad138c2cc368: Log is configured to *not* fsync() on all Append() calls
I20260811 18:55:43.549415 12745 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc353283acca46a9940dad138c2cc368: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 18:55:43.549757 12745 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc353283acca46a9940dad138c2cc368: Bootstrap complete.
I20260811 18:55:43.553864 12745 raft_consensus.cc:348] T 00000000000000000000000000000000 P bc353283acca46a9940dad138c2cc368 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc353283acca46a9940dad138c2cc368" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 33277 } }
I20260811 18:55:43.554248 12745 raft_consensus.cc:729] T 00000000000000000000000000000000 P bc353283acca46a9940dad138c2cc368 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc353283acca46a9940dad138c2cc368, State: Initialized, Role: FOLLOWER
I20260811 18:55:43.554735 12745 consensus_queue.cc:260] T 00000000000000000000000000000000 P bc353283acca46a9940dad138c2cc368 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc353283acca46a9940dad138c2cc368" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 33277 } }
I20260811 18:55:43.554855 12745 raft_consensus.cc:388] T 00000000000000000000000000000000 P bc353283acca46a9940dad138c2cc368 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:55:43.554919 12745 raft_consensus.cc:482] T 00000000000000000000000000000000 P bc353283acca46a9940dad138c2cc368 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:55:43.555011 12745 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bc353283acca46a9940dad138c2cc368 [term 1 FOLLOWER]: Advancing to term 2
I20260811 18:55:43.556854 12745 raft_consensus.cc:504] T 00000000000000000000000000000000 P bc353283acca46a9940dad138c2cc368 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc353283acca46a9940dad138c2cc368" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 33277 } }
I20260811 18:55:43.557185 12745 leader_election.cc:302] T 00000000000000000000000000000000 P bc353283acca46a9940dad138c2cc368 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bc353283acca46a9940dad138c2cc368; no voters:
I20260811 18:55:43.557482 12745 leader_election.cc:288] T 00000000000000000000000000000000 P bc353283acca46a9940dad138c2cc368 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 18:55:43.557590 12749 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bc353283acca46a9940dad138c2cc368 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 18:55:43.557799 12749 raft_consensus.cc:686] T 00000000000000000000000000000000 P bc353283acca46a9940dad138c2cc368 [term 2 LEADER]: Becoming Leader. State: Replica: bc353283acca46a9940dad138c2cc368, State: Running, Role: LEADER
I20260811 18:55:43.558096 12749 consensus_queue.cc:237] T 00000000000000000000000000000000 P bc353283acca46a9940dad138c2cc368 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc353283acca46a9940dad138c2cc368" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 33277 } }
I20260811 18:55:43.558693 12745 sys_catalog.cc:564] T 00000000000000000000000000000000 P bc353283acca46a9940dad138c2cc368 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:55:43.559024 12749 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc353283acca46a9940dad138c2cc368 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bc353283acca46a9940dad138c2cc368. Latest consensus state: current_term: 2 leader_uuid: "bc353283acca46a9940dad138c2cc368" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc353283acca46a9940dad138c2cc368" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 33277 } } }
I20260811 18:55:43.559142 12749 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc353283acca46a9940dad138c2cc368 [sys.catalog]: This master's current role is: LEADER
I20260811 18:55:43.559525 12751 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc353283acca46a9940dad138c2cc368 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "bc353283acca46a9940dad138c2cc368" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc353283acca46a9940dad138c2cc368" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 33277 } } }
I20260811 18:55:43.559686 12751 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc353283acca46a9940dad138c2cc368 [sys.catalog]: This master's current role is: LEADER
I20260811 18:55:43.560843 12755 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:55:43.563830 12755 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:55:43.564677 12755 catalog_manager.cc:1259] Loaded cluster ID: 7294f82406c7403a8adbf251def556b9
I20260811 18:55:43.564739 12755 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:55:43.567106 12755 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:55:43.567781 12755 catalog_manager.cc:5879] T 00000000000000000000000000000000 P bc353283acca46a9940dad138c2cc368: Loaded TSK: 0
I20260811 18:55:43.568971 12755 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 18:55:43.636693 12747 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:55:43.636937 12747 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:55:43.636994 12747 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:55:43.640888 12747 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:55:43.640945 12747 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:55:43.641023 12747 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.180.129
I20260811 18:55:43.646498 12747 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:35755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/krb5kdc/oryx_127.27.180.129.keytab
--principal=oryx/127.27.180.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.180.129:45869
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.180.129
--webserver_port=40035
--tserver_master_addrs=127.27.180.190:33277
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12747
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.180.129
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:55:43.647512 12747 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:55:43.648489 12747 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:55:43.658749 12792 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:55:43.658839 12747 server_base.cc:1034] running on GCE node
W20260811 18:55:43.658756 12794 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:43.658764 12797 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:55:43.659466 12747 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:55:43.660096 12747 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:55:43.661278 12747 hybrid_clock.cc:648] HybridClock initialized: now 1786474543661255 us; error 27 us; skew 500 ppm
Aug 11 18:55:43 dist-test-slave-2j7r krb5kdc[11605](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474543, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.180.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:55:43.663815 12747 init.cc:377] Logged in from keytab as oryx/127.27.180.129@KRBTEST.COM (short username oryx)
I20260811 18:55:43.664693 12747 webserver.cc:460] Webserver started at http://127.27.180.129:40035/ using document root <none> and password file <none>
I20260811 18:55:43.665243 12747 fs_manager.cc:359] Metadata directory not provided
I20260811 18:55:43.665328 12747 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:55:43.668936 12747 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260811 18:55:43.670915 12811 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:43.672058 12747 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 18:55:43.672159 12747 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/ts-0/wal
uuid: "fbd4829ec716495e937f131fa96196a9"
format_stamp: "Formatted at 2026-08-11 18:55:42 on dist-test-slave-2j7r"
server_key: "5dd9e92532e63fe44b288dd95401939c"
server_key_iv: "c93016499851fd6de2e5a46c8cb1f333"
server_key_version: "encryptionkey@0"
I20260811 18:55:43.672677 12747 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:55:43.685029 12747 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:55:43.687916 12747 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:55:43.688076 12747 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:55:43.688627 12747 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:55:43.689446 12747 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:55:43.689493 12747 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:43.689532 12747 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:55:43.689560 12747 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:43.699154 12747 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.129:45869
I20260811 18:55:43.699266 13025 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.129:45869 every 8 connection(s)
I20260811 18:55:43.700129 12747 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/ts-0/data/info.pb
I20260811 18:55:43.700987 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 12747
I20260811 18:55:43.701592 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.180.130:39039
--local_ip_for_outbound_sockets=127.27.180.130
--tserver_master_addrs=127.27.180.190:33277
--webserver_port=36351
--webserver_interface=127.27.180.130
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.130.keytab
--principal=kudu/127.27.180.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:35755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.27.180.130
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/krb5kdc/oryx_127.27.180.130.keytab with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 18:55:43 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.129@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:55:43 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.129@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:55:43.714076 13030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:43.702268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33277 (local address 127.27.180.129:39323)
0811 18:55:43.702831 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:43.702887 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:43.704190 (+ 1303us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:43.705330 (+ 1140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:43.705339 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:43.705777 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:43.706421 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:43.706440 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:43.708647 (+ 2207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:43.708650 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:43.709562 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:43.709572 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:43.710547 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:43.710549 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:43.710684 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:43.711201 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:43.711217 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:43.713394 (+ 2177us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.180.190:33277: Server oryx/127.27.180.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":128,"threads_started":1}
W20260811 18:55:43.714478 13033 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:55:43.702734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.129:39323 (local address 127.27.180.190:33277)
0811 18:55:43.703226 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:43.703240 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:43.703275 (+ 35us) server_negotiation.cc:407] Connection header received
0811 18:55:43.704588 (+ 1313us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:43.704599 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:43.704892 (+ 293us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:43.706667 (+ 1775us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:43.706994 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:43.708494 (+ 1500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:43.709679 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:43.710411 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:43.710582 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:43.713656 (+ 3074us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.180.129:39323: BlockingRecv error: recv got EOF from 127.27.180.129:39323 (error 108)
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":99,"threads_started":1}
W20260811 18:55:43.714600 13027 heartbeater.cc:644] Failed to heartbeat to 127.27.180.190:33277 (0 consecutive failures): Not authorized: Failed to ping master at 127.27.180.190:33277: Client connection negotiation failed: client connection to 127.27.180.190:33277: Server oryx/127.27.180.190@KRBTEST.COM not found in Kerberos database
Aug 11 18:55:43 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.129@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:55:43 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.129@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:55:43.719799 13030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:43.715061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33277 (local address 127.27.180.129:56227)
0811 18:55:43.715212 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:43.715226 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:43.715288 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:43.715472 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:43.715474 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:43.715590 (+ 116us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:43.715673 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:43.715677 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:43.716628 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:43.716630 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:43.717217 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:43.717228 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:43.717750 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:43.717752 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:43.717830 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:43.718222 (+ 392us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:43.718233 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:43.719703 (+ 1470us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.180.190:33277: Server oryx/127.27.180.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":94}
W20260811 18:55:43.719954 13033 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:55:43.715170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.129:56227 (local address 127.27.180.190:33277)
0811 18:55:43.715279 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:43.715284 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:43.715297 (+ 13us) server_negotiation.cc:407] Connection header received
0811 18:55:43.715336 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:43.715339 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:43.715380 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:43.715452 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:43.715759 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:43.716538 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:43.717321 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:43.717664 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:43.717758 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:43.719853 (+ 2095us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.180.129:56227: BlockingRecv error: recv got EOF from 127.27.180.129:56227 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 11 18:55:43 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.129@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:55:43 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.129@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:55:43.724821 13030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:43.720279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33277 (local address 127.27.180.129:33231)
0811 18:55:43.720422 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:43.720434 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:43.720491 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:43.720685 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:43.720686 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:43.720815 (+ 129us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:43.720882 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:43.720886 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:43.721860 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:43.721864 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:43.722365 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:43.722373 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:43.722892 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:43.722895 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:43.722947 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:43.723319 (+ 372us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:43.723330 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:43.724742 (+ 1412us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.180.190:33277: Server oryx/127.27.180.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":88}
W20260811 18:55:43.724978 13033 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:55:43.720365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.129:33231 (local address 127.27.180.190:33277)
0811 18:55:43.720464 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:43.720468 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:43.720480 (+ 12us) server_negotiation.cc:407] Connection header received
0811 18:55:43.720561 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:43.720564 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:43.720602 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:43.720669 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:43.720962 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:43.721741 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:43.722470 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:43.722801 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:43.722892 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:43.724883 (+ 1991us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.180.129:33231: BlockingRecv error: recv got EOF from 127.27.180.129:33231 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260811 18:55:43.725199 13027 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 18:55:43.814541 13035 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:55:43.814771 13035 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:55:43.814810 13035 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:55:43.818729 13035 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:55:43.818778 13035 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:55:43.818832 13035 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.180.130
I20260811 18:55:43.823885 13035 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:35755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/krb5kdc/oryx_127.27.180.130.keytab
--principal=oryx/127.27.180.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.180.130:39039
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.180.130
--webserver_port=36351
--tserver_master_addrs=127.27.180.190:33277
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13035
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.180.130
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:55:43.824772 13035 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:55:43.825680 13035 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:55:43.834578 13050 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:43.834613 13046 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:55:43.834785 13035 server_base.cc:1034] running on GCE node
W20260811 18:55:43.834921 13045 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:55:43.835301 13035 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:55:43.835893 13035 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:55:43.837071 13035 hybrid_clock.cc:648] HybridClock initialized: now 1786474543837048 us; error 35 us; skew 500 ppm
Aug 11 18:55:43 dist-test-slave-2j7r krb5kdc[11605](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474543, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.180.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:55:43.839636 13035 init.cc:377] Logged in from keytab as oryx/127.27.180.130@KRBTEST.COM (short username oryx)
I20260811 18:55:43.840544 13035 webserver.cc:460] Webserver started at http://127.27.180.130:36351/ using document root <none> and password file <none>
I20260811 18:55:43.841076 13035 fs_manager.cc:359] Metadata directory not provided
I20260811 18:55:43.841197 13035 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:55:43.844810 13035 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260811 18:55:43.846801 13065 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:43.847930 13035 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 18:55:43.848037 13035 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/ts-1/wal
uuid: "a00e8c38d9e049089b8a735997cf8777"
format_stamp: "Formatted at 2026-08-11 18:55:42 on dist-test-slave-2j7r"
server_key: "c5917a56e31bc1cd9826b818cbeed678"
server_key_iv: "592ff44ecfdd32db2147bcf17618803e"
server_key_version: "encryptionkey@0"
I20260811 18:55:43.848542 13035 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:55:43.861460 13035 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:55:43.863955 13035 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:55:43.864114 13035 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:55:43.864694 13035 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:55:43.865513 13035 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:55:43.865561 13035 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:43.865612 13035 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:55:43.865640 13035 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:43.874444 13035 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.130:39039
I20260811 18:55:43.874470 13275 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.130:39039 every 8 connection(s)
I20260811 18:55:43.875396 13035 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/ts-1/data/info.pb
I20260811 18:55:43.879209 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 13035
I20260811 18:55:43.879637 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.180.131:36715
--local_ip_for_outbound_sockets=127.27.180.131
--tserver_master_addrs=127.27.180.190:33277
--webserver_port=37449
--webserver_interface=127.27.180.131
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.131.keytab
--principal=kudu/127.27.180.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:35755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.27.180.131
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/krb5kdc/oryx_127.27.180.131.keytab with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 18:55:43 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.130@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:55:43 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.130@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
W20260811 18:55:43.886448 13033 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:55:43.877205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.130:37305 (local address 127.27.180.190:33277)
0811 18:55:43.877377 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:43.877380 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:43.878190 (+ 810us) server_negotiation.cc:407] Connection header received
0811 18:55:43.879472 (+ 1282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:43.879476 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:43.879537 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:43.879609 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:43.880856 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:43.881600 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:43.882717 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:43.882975 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:43.883070 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:43.886306 (+ 3236us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.180.130:37305: BlockingRecv error: recv got EOF from 127.27.180.130:37305 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
I20260811 18:55:43.887003 13279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:43.877580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33277 (local address 127.27.180.130:37305)
0811 18:55:43.878036 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:43.878077 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:43.879253 (+ 1176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:43.879802 (+ 549us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:43.879811 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:43.880243 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:43.880708 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:43.880721 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:43.881740 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:43.881743 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:43.882598 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:43.882608 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:43.883140 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:43.883145 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:43.883306 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:43.883820 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:43.883835 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:43.886157 (+ 2322us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.180.190:33277: Server oryx/127.27.180.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":125,"threads_started":1}
W20260811 18:55:43.887460 13277 heartbeater.cc:644] Failed to heartbeat to 127.27.180.190:33277 (0 consecutive failures): Not authorized: Failed to ping master at 127.27.180.190:33277: Client connection negotiation failed: client connection to 127.27.180.190:33277: Server oryx/127.27.180.190@KRBTEST.COM not found in Kerberos database
Aug 11 18:55:43 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.130@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:55:43 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.130@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:55:43.892625 13279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:43.887885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33277 (local address 127.27.180.130:42979)
0811 18:55:43.888001 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:43.888016 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:43.888092 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:43.888307 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:43.888310 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:43.888465 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:43.888551 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:43.888557 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:43.889310 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:43.889312 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:43.889840 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:43.889848 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:43.890261 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:43.890264 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:43.890350 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:43.890781 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:43.890791 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:43.892464 (+ 1673us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.180.190:33277: Server oryx/127.27.180.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":42}
W20260811 18:55:43.892762 13033 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:55:43.887954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.130:42979 (local address 127.27.180.190:33277)
0811 18:55:43.888062 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:43.888066 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:43.888078 (+ 12us) server_negotiation.cc:407] Connection header received
0811 18:55:43.888173 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:43.888176 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:43.888218 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:43.888280 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:43.888659 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:43.889200 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:43.889950 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:43.890164 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:43.890250 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:43.892656 (+ 2406us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.180.130:42979: BlockingRecv error: recv got EOF from 127.27.180.130:42979 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 11 18:55:43 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.130@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:55:43 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.130@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:55:43.897934 13279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:43.893062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33277 (local address 127.27.180.130:49291)
0811 18:55:43.893356 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:43.893369 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:43.893466 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:43.893668 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:43.893670 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:43.893789 (+ 119us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:43.893855 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:43.893859 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:43.894803 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:43.894805 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:43.895292 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:43.895299 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:43.895786 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:43.895788 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:43.895840 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:43.896220 (+ 380us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:43.896230 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:43.897847 (+ 1617us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.180.190:33277: Server oryx/127.27.180.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":83,"threads_started":1}
W20260811 18:55:43.898101 13033 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:55:43.893164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.130:49291 (local address 127.27.180.190:33277)
0811 18:55:43.893281 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:43.893285 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:43.893432 (+ 147us) server_negotiation.cc:407] Connection header received
0811 18:55:43.893530 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:43.893533 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:43.893575 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:43.893646 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:43.893937 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:43.894714 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:43.895375 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:43.895703 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:43.895793 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:43.897996 (+ 2203us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.180.130:49291: BlockingRecv error: recv got EOF from 127.27.180.130:49291 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260811 18:55:43.898098 13277 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 18:55:43.990413 13280 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:55:43.990682 13280 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:55:43.990725 13280 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:55:43.994745 13280 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:55:43.994791 13280 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:55:43.994860 13280 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.180.131
I20260811 18:55:43.999876 13280 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:35755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/krb5kdc/oryx_127.27.180.131.keytab
--principal=oryx/127.27.180.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.180.131:36715
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.180.131
--webserver_port=37449
--tserver_master_addrs=127.27.180.190:33277
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13280
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.180.131
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:55:44.000819 13280 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:55:44.001791 13280 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:55:44.010836 13294 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:55:44.011022 13280 server_base.cc:1034] running on GCE node
W20260811 18:55:44.010841 13293 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:44.010841 13296 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:55:44.011554 13280 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:55:44.012126 13280 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:55:44.013280 13280 hybrid_clock.cc:648] HybridClock initialized: now 1786474544013262 us; error 31 us; skew 500 ppm
Aug 11 18:55:44 dist-test-slave-2j7r krb5kdc[11605](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474544, etypes {rep=17 tkt=17 ses=17}, oryx/127.27.180.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:55:44.015812 13280 init.cc:377] Logged in from keytab as oryx/127.27.180.131@KRBTEST.COM (short username oryx)
I20260811 18:55:44.016587 13280 webserver.cc:460] Webserver started at http://127.27.180.131:37449/ using document root <none> and password file <none>
I20260811 18:55:44.017066 13280 fs_manager.cc:359] Metadata directory not provided
I20260811 18:55:44.017177 13280 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:55:44.020655 13280 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260811 18:55:44.022567 13306 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:44.023669 13280 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 18:55:44.023758 13280 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/ts-2/wal
uuid: "16496d53dfae4c21bf26cd760dd9487f"
format_stamp: "Formatted at 2026-08-11 18:55:43 on dist-test-slave-2j7r"
server_key: "e26cb700a92907f9f88352968e6a78a9"
server_key_iv: "6dd4b84ea6a7295ec42eaafa908769c9"
server_key_version: "encryptionkey@0"
I20260811 18:55:44.024230 13280 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:55:44.045699 13280 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:55:44.048585 13280 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:55:44.048727 13280 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:55:44.049256 13280 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:55:44.050014 13280 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:55:44.050058 13280 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:44.050096 13280 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:55:44.050117 13280 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:44.058524 13280 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.131:36715
I20260811 18:55:44.058552 13515 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.131:36715 every 8 connection(s)
I20260811 18:55:44.059484 13280 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786474496157106-28370-0/minicluster-data/ts-2/data/info.pb
I20260811 18:55:44.066741 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 13280
Aug 11 18:55:44 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.131@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:55:44 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.131@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
W20260811 18:55:44.073344 13033 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:55:44.061227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.131:35291 (local address 127.27.180.190:33277)
0811 18:55:44.061382 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:44.061387 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:44.062286 (+ 899us) server_negotiation.cc:407] Connection header received
0811 18:55:44.063543 (+ 1257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:44.063545 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:44.063590 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:44.063661 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:44.064868 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:44.065477 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:44.066634 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:44.066915 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:44.067019 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:44.073206 (+ 6187us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.180.131:35291: BlockingRecv error: recv got EOF from 127.27.180.131:35291 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 18:55:44.074035 13520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:44.061649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33277 (local address 127.27.180.131:35291)
0811 18:55:44.062093 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:44.062140 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:44.063356 (+ 1216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:44.063821 (+ 465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:44.063827 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:44.064229 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:44.064721 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:44.064735 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:44.065611 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:44.065614 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:44.066506 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:44.066517 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:44.067064 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:44.067069 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:44.067275 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:44.067911 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:44.067933 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:44.073042 (+ 5109us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.180.190:33277: Server oryx/127.27.180.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":112,"threads_started":1}
W20260811 18:55:44.074810 13516 heartbeater.cc:644] Failed to heartbeat to 127.27.180.190:33277 (0 consecutive failures): Not authorized: Failed to ping master at 127.27.180.190:33277: Client connection negotiation failed: client connection to 127.27.180.190:33277: Server oryx/127.27.180.190@KRBTEST.COM not found in Kerberos database
I20260811 18:55:44.076380 13524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:44.067652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33678 (local address 127.27.180.190:33277)
0811 18:55:44.067891 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:44.067894 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:44.067905 (+ 11us) server_negotiation.cc:407] Connection header received
0811 18:55:44.067936 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:44.067938 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:44.067973 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:44.068048 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:44.068808 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:44.069384 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:44.070395 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:44.070689 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:44.070806 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:44.072167 (+ 1361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:44.072174 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:44.072192 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:55:44.072227 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:44.074249 (+ 2022us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:44.074720 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:44.074725 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:44.074729 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:44.074842 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:55:44.075123 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:55:44.075126 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:55:44.075128 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:55:44.075531 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:44.075735 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:44.076031 (+ 296us) server_negotiation.cc:299] Negotiation successful
0811 18:55:44.076266 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":62,"threads_started":1}
Aug 11 18:55:44 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.131@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:55:44 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.131@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:55:44.080132 13520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:44.075217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33277 (local address 127.27.180.131:44833)
0811 18:55:44.075320 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:44.075333 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:44.075404 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:44.075599 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:44.075601 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:44.075833 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:44.075923 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:44.075928 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:44.076722 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:44.076724 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:44.077355 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:44.077366 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:44.077778 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:44.077782 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:44.077871 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:44.078443 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:44.078456 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:44.080045 (+ 1589us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.180.190:33277: Server oryx/127.27.180.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":31}
W20260811 18:55:44.080262 13033 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:55:44.075289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.131:44833 (local address 127.27.180.190:33277)
0811 18:55:44.075389 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:44.075392 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:44.075404 (+ 12us) server_negotiation.cc:407] Connection header received
0811 18:55:44.075472 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:44.075475 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:44.075512 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:44.075573 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:44.076027 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:44.076624 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:44.077479 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:44.077683 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:44.077756 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:44.080158 (+ 2402us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.180.131:44833: BlockingRecv error: recv got EOF from 127.27.180.131:44833 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 11 18:55:44 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.131@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:55:44 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.131@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:55:44.084643 13520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:44.080549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33277 (local address 127.27.180.131:38847)
0811 18:55:44.080652 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:44.080663 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:44.080734 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:44.080930 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:44.080933 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:44.081064 (+ 131us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:44.081177 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:44.081182 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:44.081828 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:44.081831 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:44.082325 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:44.082332 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:44.082707 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:44.082711 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:44.082785 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:44.083115 (+ 330us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:44.083124 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:44.084535 (+ 1411us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.180.190:33277: Server oryx/127.27.180.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":43}
W20260811 18:55:44.084731 13033 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:55:44.080633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.131:38847 (local address 127.27.180.190:33277)
0811 18:55:44.080767 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:44.080771 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:44.080783 (+ 12us) server_negotiation.cc:407] Connection header received
0811 18:55:44.080812 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:44.080815 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:44.080846 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:44.080916 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:44.081279 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:44.081736 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:44.082422 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:44.082619 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:44.082699 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:44.084645 (+ 1946us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.180.131:38847: BlockingRecv error: recv got EOF from 127.27.180.131:38847 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
W20260811 18:55:44.084883 13516 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 11 18:55:44 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.129@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:55:44 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.129@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:55:44.730780 13525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:44.725726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33277 (local address 127.27.180.129:35277)
0811 18:55:44.725998 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:44.726011 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:44.726088 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:44.726310 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:44.726312 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:44.726493 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:44.726605 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:44.726611 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:44.727389 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:44.727392 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:44.727994 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:44.728003 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:44.728403 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:44.728406 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:44.728472 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:44.728893 (+ 421us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:44.728903 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:44.730669 (+ 1766us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.180.190:33277: Server oryx/127.27.180.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":85,"threads_started":1}
W20260811 18:55:44.730916 13526 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:55:44.725794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.129:35277 (local address 127.27.180.190:33277)
0811 18:55:44.726057 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:44.726061 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:44.726074 (+ 13us) server_negotiation.cc:407] Connection header received
0811 18:55:44.726167 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:44.726169 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:44.726214 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:44.726287 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:44.726688 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:44.727285 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:44.728086 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:44.728306 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:44.728379 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:44.730791 (+ 2412us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.180.129:35277: BlockingRecv error: recv got EOF from 127.27.180.129:35277 (error 108)
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":86,"threads_started":1}
Aug 11 18:55:44 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.130@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:55:44 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.130@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:55:44.903856 13527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:44.898630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33277 (local address 127.27.180.130:36553)
0811 18:55:44.898909 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:44.898922 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:44.899022 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:44.899258 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:44.899262 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:44.899451 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:44.899545 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:44.899550 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:44.900274 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:44.900278 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:44.900825 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:44.900833 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:44.901338 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:44.901342 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:44.901425 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:44.901827 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:44.901838 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:44.903687 (+ 1849us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.180.190:33277: Server oryx/127.27.180.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":81,"threads_started":1}
W20260811 18:55:44.903930 13526 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:55:44.898703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.130:36553 (local address 127.27.180.190:33277)
0811 18:55:44.898860 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:44.898865 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:44.898986 (+ 121us) server_negotiation.cc:407] Connection header received
0811 18:55:44.899106 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:44.899110 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:44.899155 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:44.899219 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:44.899653 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:44.900174 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:44.900943 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:44.901217 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:44.901297 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:44.903817 (+ 2520us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.180.130:36553: BlockingRecv error: recv got EOF from 127.27.180.130:36553 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 11 18:55:45 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.131@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:55:45 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.131@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:55:45.090137 13530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:45.085450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33277 (local address 127.27.180.131:45811)
0811 18:55:45.085686 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:45.085699 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:45.085789 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:45.086003 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:45.086005 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:45.086174 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:45.086269 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:45.086274 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:45.086946 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:45.086948 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:45.087491 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:45.087497 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:45.087845 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:45.087848 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:45.087912 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:45.088312 (+ 400us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:45.088323 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:45.090016 (+ 1693us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.180.190:33277: Server oryx/127.27.180.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":69,"threads_started":1}
W20260811 18:55:45.090266 13526 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:55:45.085502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.131:45811 (local address 127.27.180.190:33277)
0811 18:55:45.085623 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:45.085626 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:45.085753 (+ 127us) server_negotiation.cc:407] Connection header received
0811 18:55:45.085854 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:45.085856 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:45.085899 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:45.085974 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:45.086355 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:45.086871 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:45.087567 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:45.087773 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:45.087839 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:45.090148 (+ 2309us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.180.131:45811: BlockingRecv error: recv got EOF from 127.27.180.131:45811 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 11 18:55:45 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.129@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:55:45 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.129@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:55:45.736937 13533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:45.731573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33277 (local address 127.27.180.129:55749)
0811 18:55:45.731849 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:45.731862 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:45.731957 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:45.732171 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:45.732173 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:45.732341 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:45.732449 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:45.732454 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:45.733168 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:45.733170 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:45.733732 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:45.733739 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:45.734116 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:45.734119 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:45.734172 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:45.734668 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:45.734683 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:45.736782 (+ 2099us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.180.190:33277: Server oryx/127.27.180.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":91,"threads_started":1}
W20260811 18:55:45.737044 13536 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:55:45.731633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.129:55749 (local address 127.27.180.190:33277)
0811 18:55:45.731868 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:45.731872 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:45.731921 (+ 49us) server_negotiation.cc:407] Connection header received
0811 18:55:45.732023 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:45.732025 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:45.732071 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:45.732146 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:45.732529 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:45.733049 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:45.733816 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:45.734039 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:45.734109 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:45.736921 (+ 2812us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.180.129:55749: BlockingRecv error: recv got EOF from 127.27.180.129:55749 (error 108)
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":82,"threads_started":1}
Aug 11 18:55:45 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.130@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:55:45 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.130@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:55:45.909741 13539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:45.904693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33277 (local address 127.27.180.130:48831)
0811 18:55:45.904943 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:45.904956 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:45.905051 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:45.905269 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:45.905271 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:45.905456 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:45.905553 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:45.905558 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:45.906287 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:45.906290 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:45.906917 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:45.906928 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:45.907345 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:45.907348 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:45.907425 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:45.907894 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:45.907906 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:45.909625 (+ 1719us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.180.190:33277: Server oryx/127.27.180.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":67,"threads_started":1}
W20260811 18:55:45.909844 13536 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:55:45.904766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.130:48831 (local address 127.27.180.190:33277)
0811 18:55:45.904922 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:45.904926 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:45.905015 (+ 89us) server_negotiation.cc:407] Connection header received
0811 18:55:45.905148 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:45.905151 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:45.905195 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:45.905257 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:45.905630 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:45.906160 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:45.907026 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:45.907236 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:45.907310 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:45.909745 (+ 2435us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.180.130:48831: BlockingRecv error: recv got EOF from 127.27.180.130:48831 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 11 18:55:46 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.131@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:55:46 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.131@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:55:46.101341 13542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:46.090801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33277 (local address 127.27.180.131:49701)
0811 18:55:46.091085 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:46.091098 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:46.091190 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:46.091402 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:46.091404 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:46.091576 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:46.091675 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:46.091680 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:46.092438 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:46.092442 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:46.093028 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:46.093038 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:46.093518 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:46.093523 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:46.093610 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:46.099250 (+ 5640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:46.099263 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:46.101141 (+ 1878us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.180.190:33277: Server oryx/127.27.180.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":105,"threads_started":1}
W20260811 18:55:46.101395 13536 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:55:46.090922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.131:49701 (local address 127.27.180.190:33277)
0811 18:55:46.091048 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:46.091051 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:46.091152 (+ 101us) server_negotiation.cc:407] Connection header received
0811 18:55:46.091251 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:46.091253 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:46.091299 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:46.091364 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:46.091791 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:46.092320 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:46.093191 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:46.093411 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:46.093500 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:46.101295 (+ 7795us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.180.131:49701: BlockingRecv error: recv got EOF from 127.27.180.131:49701 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 11 18:55:46 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.129@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:55:46 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.129@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:55:46.742815 13543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:46.737870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33277 (local address 127.27.180.129:40883)
0811 18:55:46.738141 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:46.738155 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:46.738242 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:46.738510 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:46.738512 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:46.738684 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:46.738783 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:46.738788 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:46.739483 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:46.739485 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:46.740045 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:46.740052 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:46.740419 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:46.740421 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:46.740480 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:46.740931 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:46.740942 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:46.742684 (+ 1742us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.180.190:33277: Server oryx/127.27.180.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":75,"threads_started":1}
W20260811 18:55:46.742971 13547 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:55:46.737948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.129:40883 (local address 127.27.180.190:33277)
0811 18:55:46.738177 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:46.738180 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:46.738198 (+ 18us) server_negotiation.cc:407] Connection header received
0811 18:55:46.738355 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:46.738358 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:46.738409 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:46.738490 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:46.738862 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:46.739403 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:46.740121 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:46.740336 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:46.740401 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:46.742824 (+ 2423us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.180.129:40883: BlockingRecv error: recv got EOF from 127.27.180.129:40883 (error 108)
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":72,"threads_started":1}
Aug 11 18:55:46 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.130@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:55:46 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.130@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:55:46.915426 13549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:46.910434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33277 (local address 127.27.180.130:33597)
0811 18:55:46.910705 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:46.910719 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:46.910818 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:46.911026 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:46.911028 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:46.911206 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:46.911306 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:46.911311 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:46.911996 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:46.911998 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:46.912551 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:46.912559 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:46.912910 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:46.912912 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:46.912975 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:46.913484 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:46.913496 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:46.915290 (+ 1794us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.180.190:33277: Server oryx/127.27.180.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":102,"threads_started":1}
W20260811 18:55:46.915549 13547 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:55:46.910519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.130:33597 (local address 127.27.180.190:33277)
0811 18:55:46.910659 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:46.910663 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:46.910777 (+ 114us) server_negotiation.cc:407] Connection header received
0811 18:55:46.910895 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:46.910898 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:46.910939 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:46.911001 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:46.911394 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:46.911919 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:46.912630 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:46.912832 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:46.912907 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:46.915444 (+ 2537us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.180.130:33597: BlockingRecv error: recv got EOF from 127.27.180.130:33597 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 11 18:55:47 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.131@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:55:47 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.131@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:55:47.107209 13552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:47.102101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33277 (local address 127.27.180.131:37883)
0811 18:55:47.102404 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:47.102416 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:47.102520 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:47.102751 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:47.102754 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:47.102952 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:47.103045 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:47.103052 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:47.103789 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:47.103792 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:47.104335 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:47.104343 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:47.104766 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:47.104769 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:47.104851 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:47.105300 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:47.105310 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:47.107060 (+ 1750us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.180.190:33277: Server oryx/127.27.180.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":113,"threads_started":1}
W20260811 18:55:47.107298 13547 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:55:47.102184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.131:37883 (local address 127.27.180.190:33277)
0811 18:55:47.102329 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:47.102334 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:47.102481 (+ 147us) server_negotiation.cc:407] Connection header received
0811 18:55:47.102604 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:47.102607 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:47.102655 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:47.102720 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:47.103162 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:47.103686 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:47.104454 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:47.104670 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:47.104753 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:47.107198 (+ 2445us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.180.131:37883: BlockingRecv error: recv got EOF from 127.27.180.131:37883 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 11 18:55:47 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.129@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:55:47 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.129@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:55:47.748437 13562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:47.743500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33277 (local address 127.27.180.129:49403)
0811 18:55:47.743752 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:47.743765 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:47.743852 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:47.744068 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:47.744070 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:47.744245 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:47.744344 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:47.744349 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:47.745073 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:47.745076 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:47.745665 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:47.745676 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:47.746083 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:47.746085 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:47.746141 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:47.746566 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:47.746577 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:47.748264 (+ 1687us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.180.190:33277: Server oryx/127.27.180.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":74,"threads_started":1}
W20260811 18:55:47.748523 13565 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:55:47.743546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.129:49403 (local address 127.27.180.190:33277)
0811 18:55:47.743797 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:47.743800 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:47.743813 (+ 13us) server_negotiation.cc:407] Connection header received
0811 18:55:47.743940 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:47.743943 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:47.743985 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:47.744055 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:47.744444 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:47.744984 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:47.745760 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:47.746000 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:47.746073 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:47.748407 (+ 2334us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.180.129:49403: BlockingRecv error: recv got EOF from 127.27.180.129:49403 (error 108)
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":89,"threads_started":1}
Aug 11 18:55:47 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.130@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:55:47 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.130@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:55:47.920993 13567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:47.916107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33277 (local address 127.27.180.130:34183)
0811 18:55:47.916351 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:47.916365 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:47.916477 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:47.916681 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:47.916682 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:47.916858 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:47.916959 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:47.916964 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:47.917652 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:47.917654 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:47.918197 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:47.918204 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:47.918648 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:47.918651 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:47.918710 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:47.919182 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:47.919194 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:47.920877 (+ 1683us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.180.190:33277: Server oryx/127.27.180.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":69,"threads_started":1}
W20260811 18:55:47.921137 13565 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:55:47.916179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.130:34183 (local address 127.27.180.190:33277)
0811 18:55:47.916346 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:47.916350 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:47.916427 (+ 77us) server_negotiation.cc:407] Connection header received
0811 18:55:47.916549 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:47.916551 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:47.916594 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:47.916658 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:47.917042 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:47.917579 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:47.918323 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:47.918551 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:47.918635 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:47.921010 (+ 2375us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.180.130:34183: BlockingRecv error: recv got EOF from 127.27.180.130:34183 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 11 18:55:48 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.131@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:55:48 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.131@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:55:48.112910 13568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:48.107957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33277 (local address 127.27.180.131:46245)
0811 18:55:48.108329 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:48.108343 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:48.108442 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:48.108648 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:48.108650 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:48.108830 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:48.108931 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:48.108936 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:48.109663 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:48.109666 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:48.110233 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:48.110243 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:48.110601 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:48.110603 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:48.110662 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:48.111108 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:48.111119 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:48.112790 (+ 1671us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.180.190:33277: Server oryx/127.27.180.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":104,"threads_started":1}
W20260811 18:55:48.113029 13565 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:55:48.108065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.131:46245 (local address 127.27.180.190:33277)
0811 18:55:48.108194 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:48.108198 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:48.108402 (+ 204us) server_negotiation.cc:407] Connection header received
0811 18:55:48.108512 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:48.108514 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:48.108557 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:48.108622 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:48.109015 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:48.109551 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:48.110322 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:48.110526 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:48.110590 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:48.112928 (+ 2338us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.180.131:46245: BlockingRecv error: recv got EOF from 127.27.180.131:46245 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 11 18:55:48 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.129@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:55:48 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.129@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:55:48.754253 13569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:48.749314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33277 (local address 127.27.180.129:37715)
0811 18:55:48.749575 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:48.749589 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:48.749680 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:48.749871 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:48.749873 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:48.750056 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:48.750152 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:48.750157 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:48.750847 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:48.750849 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:48.751432 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:48.751439 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:48.751831 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:48.751834 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:48.751892 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:48.752371 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:48.752393 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:48.754135 (+ 1742us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.180.190:33277: Server oryx/127.27.180.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":79,"threads_started":1}
W20260811 18:55:48.754400 13570 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:55:48.749349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.129:37715 (local address 127.27.180.190:33277)
0811 18:55:48.749573 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:48.749577 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:48.749643 (+ 66us) server_negotiation.cc:407] Connection header received
0811 18:55:48.749745 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:48.749747 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:48.749789 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:48.749863 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:48.750231 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:48.750769 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:48.751517 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:48.751742 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:48.751821 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:48.754267 (+ 2446us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.180.129:37715: BlockingRecv error: recv got EOF from 127.27.180.129:37715 (error 108)
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":71,"threads_started":1}
Aug 11 18:55:48 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.130@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:55:48 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.130@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:55:48.926687 13571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:48.921731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33277 (local address 127.27.180.130:35239)
0811 18:55:48.922000 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:48.922013 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:48.922104 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:48.922303 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:48.922305 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:48.922491 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:48.922586 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:48.922591 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:48.923388 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:48.923392 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:48.923943 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:48.923952 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:48.924333 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:48.924336 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:48.924403 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:48.924839 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:48.924849 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:48.926575 (+ 1726us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.180.190:33277: Server oryx/127.27.180.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":76,"threads_started":1}
W20260811 18:55:48.926831 13570 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:55:48.921857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.130:35239 (local address 127.27.180.190:33277)
0811 18:55:48.922029 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:48.922034 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:48.922063 (+ 29us) server_negotiation.cc:407] Connection header received
0811 18:55:48.922175 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:48.922177 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:48.922218 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:48.922281 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:48.922723 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:48.923268 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:48.924030 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:48.924253 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:48.924326 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:48.926715 (+ 2389us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.180.130:35239: BlockingRecv error: recv got EOF from 127.27.180.130:35239 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
Aug 11 18:55:49 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.131@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:55:49 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.131@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:55:49.118762 13574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:49.113730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33277 (local address 127.27.180.131:40239)
0811 18:55:49.114033 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:49.114046 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:49.114139 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:49.114348 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:49.114350 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:49.114535 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:49.114633 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:49.114638 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:49.115331 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:49.115333 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:49.115891 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:49.115899 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:49.116283 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:49.116285 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:49.116343 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:49.116824 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:49.116837 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:49.118612 (+ 1775us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.180.190:33277: Server oryx/127.27.180.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":84,"threads_started":1}
W20260811 18:55:49.118837 13570 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:55:49.113818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.131:40239 (local address 127.27.180.190:33277)
0811 18:55:49.113969 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:49.113973 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:49.114106 (+ 133us) server_negotiation.cc:407] Connection header received
0811 18:55:49.114205 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:49.114207 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:49.114254 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:49.114318 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:49.114715 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:49.115240 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:49.115983 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:49.116206 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:49.116273 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:49.118740 (+ 2467us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.180.131:40239: BlockingRecv error: recv got EOF from 127.27.180.131:40239 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 11 18:55:49 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.129@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:55:49 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.129@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:55:49.760252 13575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:49.754927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33277 (local address 127.27.180.129:43439)
0811 18:55:49.755189 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:49.755202 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:49.755322 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:49.755596 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:49.755600 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:49.755825 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:49.755970 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:49.755979 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:49.756847 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:49.756850 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:49.757469 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:49.757479 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:49.757851 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:49.757853 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:49.757920 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:49.758349 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:49.758360 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:49.760136 (+ 1776us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.180.190:33277: Server oryx/127.27.180.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":73,"threads_started":1}
W20260811 18:55:49.760387 13576 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:55:49.754992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.129:43439 (local address 127.27.180.190:33277)
0811 18:55:49.755225 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:49.755228 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:49.755260 (+ 32us) server_negotiation.cc:407] Connection header received
0811 18:55:49.755415 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:49.755418 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:49.755475 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:49.755563 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:49.756093 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:49.756745 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:49.757557 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:49.757775 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:49.757842 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:49.760256 (+ 2414us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.180.129:43439: BlockingRecv error: recv got EOF from 127.27.180.129:43439 (error 108)
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":69,"threads_started":1}
Aug 11 18:55:49 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.130@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:55:49 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.130@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:55:49.932861 13578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:49.927440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33277 (local address 127.27.180.130:60665)
0811 18:55:49.927710 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:49.927724 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:49.927822 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:49.928044 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:49.928046 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:49.928249 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:49.928366 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:49.928372 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:49.929211 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:49.929214 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:49.929799 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:49.929810 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:49.930366 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:49.930369 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:49.930450 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:49.930978 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:49.930991 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:49.932732 (+ 1741us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.180.190:33277: Server oryx/127.27.180.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":102,"threads_started":1}
W20260811 18:55:49.932999 13576 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:55:49.927501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.130:60665 (local address 127.27.180.190:33277)
0811 18:55:49.927625 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:49.927628 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:49.927786 (+ 158us) server_negotiation.cc:407] Connection header received
0811 18:55:49.927913 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:49.927915 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:49.927960 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:49.928024 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:49.928501 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:49.929053 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:49.929989 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:49.930228 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:49.930321 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:49.932894 (+ 2573us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.180.130:60665: BlockingRecv error: recv got EOF from 127.27.180.130:60665 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 11 18:55:50 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.131@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:55:50 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.131@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:55:50.124430 13579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:50.119540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33277 (local address 127.27.180.131:34177)
0811 18:55:50.119808 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:50.119821 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:50.119929 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:50.120139 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:50.120141 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:50.120326 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:50.120423 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:50.120428 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:50.121141 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:50.121144 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:50.121685 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:50.121693 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:50.122073 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:50.122076 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:50.122131 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:50.122568 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:50.122579 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:50.124309 (+ 1730us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.180.190:33277: Server oryx/127.27.180.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":84,"threads_started":1}
W20260811 18:55:50.124550 13576 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:55:50.119613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.131:34177 (local address 127.27.180.190:33277)
0811 18:55:50.119736 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:50.119740 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:50.119890 (+ 150us) server_negotiation.cc:407] Connection header received
0811 18:55:50.120001 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:50.120004 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:50.120048 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:50.120112 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:50.120505 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:50.121034 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:50.121772 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:50.121994 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:50.122064 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:50.124447 (+ 2383us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.180.131:34177: BlockingRecv error: recv got EOF from 127.27.180.131:34177 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 11 18:55:50 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.129@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:55:50 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.129@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:55:50.765974 13584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:50.760953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33277 (local address 127.27.180.129:40277)
0811 18:55:50.761349 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:50.761363 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:50.761470 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:50.761676 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:50.761678 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:50.761864 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:50.761967 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:50.761973 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:50.762652 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:50.762654 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:50.763196 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:50.763202 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:50.763552 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:50.763554 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:50.763609 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:50.764056 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:50.764068 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:50.765826 (+ 1758us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.180.190:33277: Server oryx/127.27.180.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":109,"threads_started":1}
W20260811 18:55:50.766110 13585 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:55:50.761032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.129:40277 (local address 127.27.180.190:33277)
0811 18:55:50.761309 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:50.761313 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:50.761424 (+ 111us) server_negotiation.cc:407] Connection header received
0811 18:55:50.761538 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:50.761540 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:50.761583 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:50.761653 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:50.762051 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:50.762575 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:50.763272 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:50.763479 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:50.763543 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:50.765979 (+ 2436us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.180.129:40277: BlockingRecv error: recv got EOF from 127.27.180.129:40277 (error 108)
Metrics: {"server-negotiator.queue_time_us":202,"thread_start_us":90,"threads_started":1}
Aug 11 18:55:50 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.130@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:55:50 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.130@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:55:50.938781 13587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:50.933552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33277 (local address 127.27.180.130:60207)
0811 18:55:50.933884 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:50.933897 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:50.933994 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:50.934199 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:50.934201 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:50.934397 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:50.934495 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:50.934501 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:50.935250 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:50.935252 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:50.935804 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:50.935811 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:50.936184 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:50.936187 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:50.936243 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:50.936705 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:50.936717 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:50.938644 (+ 1927us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.180.190:33277: Server oryx/127.27.180.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":96,"threads_started":1}
W20260811 18:55:50.938908 13585 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:55:50.933663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.130:60207 (local address 127.27.180.190:33277)
0811 18:55:50.933824 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:50.933828 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:50.933955 (+ 127us) server_negotiation.cc:407] Connection header received
0811 18:55:50.934062 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:50.934064 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:50.934108 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:50.934177 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:50.934603 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:50.935150 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:50.935884 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:50.936103 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:50.936174 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:50.938794 (+ 2620us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.180.130:60207: BlockingRecv error: recv got EOF from 127.27.180.130:60207 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 11 18:55:51 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.131@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:55:51 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.131@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:55:51.134655 13588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:51.125209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33277 (local address 127.27.180.131:33055)
0811 18:55:51.125537 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:51.125551 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:51.125646 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:51.125839 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:51.125841 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:51.126053 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:51.126151 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:51.126157 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:51.126868 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:51.126870 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:51.127452 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:51.127462 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:51.127886 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:51.127890 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:51.127983 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:51.132701 (+ 4718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:51.132714 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:51.134506 (+ 1792us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.180.190:33277: Server oryx/127.27.180.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":100,"threads_started":1}
W20260811 18:55:51.134754 13585 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:55:51.125241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.131:33055 (local address 127.27.180.190:33277)
0811 18:55:51.125397 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:51.125401 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:51.125609 (+ 208us) server_negotiation.cc:407] Connection header received
0811 18:55:51.125707 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:51.125709 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:51.125754 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:51.125814 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:51.126236 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:51.126790 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:51.127563 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:51.127781 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:51.127870 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:51.134656 (+ 6786us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.180.131:33055: BlockingRecv error: recv got EOF from 127.27.180.131:33055 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 11 18:55:51 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.129@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:55:51 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.129@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:55:51.771759 13593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:51.766646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33277 (local address 127.27.180.129:50167)
0811 18:55:51.766926 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:51.766942 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:51.767033 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:51.767266 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:51.767268 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:51.767460 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:51.767560 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:51.767565 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:51.768294 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:51.768296 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:51.768844 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:51.768850 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:51.769259 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:51.769261 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:51.769316 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:51.769818 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:51.769829 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:51.771644 (+ 1815us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.180.190:33277: Server oryx/127.27.180.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":69,"threads_started":1}
W20260811 18:55:51.771914 13596 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:55:51.766792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.129:50167 (local address 127.27.180.190:33277)
0811 18:55:51.767065 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:51.767068 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:51.767081 (+ 13us) server_negotiation.cc:407] Connection header received
0811 18:55:51.767115 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:51.767117 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:51.767158 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:51.767237 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:51.767654 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:51.768214 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:51.768933 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:51.769179 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:51.769260 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:51.771781 (+ 2521us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.180.129:50167: BlockingRecv error: recv got EOF from 127.27.180.129:50167 (error 108)
Metrics: {"server-negotiator.queue_time_us":197,"thread_start_us":85,"threads_started":1}
Aug 11 18:55:51 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.130@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:55:51 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.130@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:55:51.945295 13597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:51.939650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33277 (local address 127.27.180.130:55611)
0811 18:55:51.939958 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:51.939973 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:51.940084 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:51.940323 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:51.940326 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:51.940526 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:51.940622 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:51.940628 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:51.941588 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:51.941591 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:51.942196 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:51.942204 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:51.942649 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:51.942652 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:51.942733 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:51.943186 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:51.943196 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:51.945122 (+ 1926us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.180.190:33277: Server oryx/127.27.180.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":91,"threads_started":1}
W20260811 18:55:51.945514 13596 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:55:51.939750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.130:55611 (local address 127.27.180.190:33277)
0811 18:55:51.939916 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:51.939921 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:51.940035 (+ 114us) server_negotiation.cc:407] Connection header received
0811 18:55:51.940167 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:51.940171 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:51.940221 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:51.940289 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:51.940728 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:51.941476 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:51.942303 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:51.942546 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:51.942632 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:51.945260 (+ 2628us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.180.130:55611: BlockingRecv error: recv got EOF from 127.27.180.130:55611 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 11 18:55:52 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.131@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:55:52 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.131@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:55:52.140589 13599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:52.135392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33277 (local address 127.27.180.131:42435)
0811 18:55:52.135719 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:52.135731 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:52.135850 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:52.136101 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:52.136103 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:52.136322 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:52.136458 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:52.136466 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:52.137277 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:52.137281 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:52.137838 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:52.137846 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:52.138253 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:52.138257 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:52.138337 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:52.138781 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:52.138792 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:52.140440 (+ 1648us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.180.190:33277: Server oryx/127.27.180.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":104,"threads_started":1}
W20260811 18:55:52.140678 13596 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:55:52.135494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.131:42435 (local address 127.27.180.190:33277)
0811 18:55:52.135631 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:52.135636 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:52.135806 (+ 170us) server_negotiation.cc:407] Connection header received
0811 18:55:52.135948 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:52.135951 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:52.135996 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:52.136059 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:52.136566 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:52.137178 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:52.137946 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:52.138157 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:52.138240 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:52.140581 (+ 2341us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.180.131:42435: BlockingRecv error: recv got EOF from 127.27.180.131:42435 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 11 18:55:52 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.129@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:55:52 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.129@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:55:52.778091 13600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:52.772529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33277 (local address 127.27.180.129:38919)
0811 18:55:52.772823 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:52.772836 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:52.772947 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:52.773274 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:52.773378 (+ 104us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:52.773604 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:52.773713 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:52.773720 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:52.774501 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:52.774505 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:52.775069 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:52.775077 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:52.775537 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:52.775540 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:52.775629 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:52.776127 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:52.776142 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:52.777973 (+ 1831us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.180.190:33277: Server oryx/127.27.180.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":85,"threads_started":1}
W20260811 18:55:52.778239 13602 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:55:52.772579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.129:38919 (local address 127.27.180.190:33277)
0811 18:55:52.772858 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:52.772863 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:52.772880 (+ 17us) server_negotiation.cc:407] Connection header received
0811 18:55:52.773042 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:52.773045 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:52.773125 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:52.773237 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:52.773833 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:52.774390 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:52.775177 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:52.775409 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:52.775517 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:52.778105 (+ 2588us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.180.129:38919: BlockingRecv error: recv got EOF from 127.27.180.129:38919 (error 108)
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":90,"threads_started":1}
Aug 11 18:55:52 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.130@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:55:52 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.130@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:55:52.951033 13603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:52.946088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33277 (local address 127.27.180.130:34259)
0811 18:55:52.946339 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:52.946353 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:52.946464 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:52.946683 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:52.946685 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:52.946858 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:52.946969 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:52.946976 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:52.947696 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:52.947699 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:52.948250 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:52.948257 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:52.948620 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:52.948623 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:52.948682 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:52.949133 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:52.949144 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:52.950894 (+ 1750us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.180.190:33277: Server oryx/127.27.180.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":73,"threads_started":1}
W20260811 18:55:52.951133 13602 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:55:52.946156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.130:34259 (local address 127.27.180.190:33277)
0811 18:55:52.946322 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:52.946327 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:52.946415 (+ 88us) server_negotiation.cc:407] Connection header received
0811 18:55:52.946538 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:52.946541 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:52.946592 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:52.946670 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:52.947065 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:52.947612 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:52.948330 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:52.948545 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:52.948611 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:52.951035 (+ 2424us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.180.130:34259: BlockingRecv error: recv got EOF from 127.27.180.130:34259 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 11 18:55:53 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.131@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:55:53 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.131@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:55:53.146476 13604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:53.141367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33277 (local address 127.27.180.131:36719)
0811 18:55:53.141630 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:53.141643 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:53.141741 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:53.141952 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:53.141954 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:53.142130 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:53.142232 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:53.142237 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:53.142927 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:53.142930 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:53.143471 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:53.143478 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:53.143847 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:53.143849 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:53.143918 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:53.144348 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:53.144360 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:53.146343 (+ 1983us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.180.190:33277: Server oryx/127.27.180.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":73,"threads_started":1}
W20260811 18:55:53.146582 13602 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:55:53.141424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.131:36719 (local address 127.27.180.190:33277)
0811 18:55:53.141557 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:53.141560 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:53.141703 (+ 143us) server_negotiation.cc:407] Connection header received
0811 18:55:53.141808 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:53.141810 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:53.141855 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:53.141928 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:53.142313 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:53.142828 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:53.143557 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:53.143769 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:53.143838 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:53.146475 (+ 2637us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.180.131:36719: BlockingRecv error: recv got EOF from 127.27.180.131:36719 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 11 18:55:53 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.129@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:55:53 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.129@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:55:53.784153 13607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:53.778810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33277 (local address 127.27.180.129:37087)
0811 18:55:53.779087 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:53.779100 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:53.779197 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:53.779401 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:53.779415 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:53.779594 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:53.779696 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:53.779701 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:53.780422 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:53.780425 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:53.781246 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:53.781257 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:53.781752 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:53.781755 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:53.781826 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:53.782270 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:53.782281 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:53.784039 (+ 1758us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.180.190:33277: Server oryx/127.27.180.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":87,"threads_started":1}
W20260811 18:55:53.784302 13609 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:55:53.778873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.129:37087 (local address 127.27.180.190:33277)
0811 18:55:53.779107 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:53.779111 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:53.779162 (+ 51us) server_negotiation.cc:407] Connection header received
0811 18:55:53.779264 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:53.779266 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:53.779307 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:53.779390 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:53.779781 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:53.780314 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:53.781351 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:53.781637 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:53.781715 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:53.784168 (+ 2453us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.180.129:37087: BlockingRecv error: recv got EOF from 127.27.180.129:37087 (error 108)
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":66,"threads_started":1}
Aug 11 18:55:53 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.130@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:55:53 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.130@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:55:53.957155 13612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:53.951861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33277 (local address 127.27.180.130:48481)
0811 18:55:53.952163 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:53.952179 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:53.952319 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:53.952526 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:53.952529 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:53.952762 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:53.952891 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:53.952898 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:53.953704 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:53.953707 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:53.954263 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:53.954271 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:53.954703 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:53.954705 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:53.954765 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:53.955241 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:53.955255 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:53.956976 (+ 1721us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.180.190:33277: Server oryx/127.27.180.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":111,"threads_started":1}
W20260811 18:55:53.957271 13609 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:55:53.951896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.130:48481 (local address 127.27.180.190:33277)
0811 18:55:53.952027 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:53.952033 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:53.952288 (+ 255us) server_negotiation.cc:407] Connection header received
0811 18:55:53.952385 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:53.952387 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:53.952442 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:53.952503 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:53.952993 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:53.953589 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:53.954382 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:53.954612 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:53.954690 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:53.957152 (+ 2462us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.180.130:48481: BlockingRecv error: recv got EOF from 127.27.180.130:48481 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 11 18:55:54 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.131@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:55:54 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.131@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:55:54.152489 13614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:54.147273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33277 (local address 127.27.180.131:47807)
0811 18:55:54.147580 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:54.147596 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:54.147715 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:54.147956 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:54.147959 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:54.148162 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:54.148290 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:54.148297 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:54.149043 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:54.149047 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:54.149612 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:54.149621 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:54.150052 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:54.150055 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:54.150138 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:54.150610 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:54.150624 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:54.152372 (+ 1748us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.180.190:33277: Server oryx/127.27.180.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":103,"threads_started":1}
W20260811 18:55:54.152619 13609 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:55:54.147439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.131:47807 (local address 127.27.180.190:33277)
0811 18:55:54.147589 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:54.147594 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:54.147678 (+ 84us) server_negotiation.cc:407] Connection header received
0811 18:55:54.147806 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:54.147809 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:54.147855 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:54.147922 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:54.148406 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:54.148933 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:54.149737 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:54.149951 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:54.150037 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:54.152507 (+ 2470us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.180.131:47807: BlockingRecv error: recv got EOF from 127.27.180.131:47807 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 11 18:55:54 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.129@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:55:54 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.129@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:55:54.790220 13617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:54.784831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33277 (local address 127.27.180.129:42583)
0811 18:55:54.785126 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:54.785139 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:54.785229 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:54.785495 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:54.785498 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:54.785701 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:54.785811 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:54.785817 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:54.786623 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:54.786626 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:54.787193 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:54.787201 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:54.787685 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:54.787688 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:54.787754 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:54.788231 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:54.788242 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:54.790089 (+ 1847us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.180.190:33277: Server oryx/127.27.180.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":87,"threads_started":1}
W20260811 18:55:54.790338 13618 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:55:54.784902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.129:42583 (local address 127.27.180.190:33277)
0811 18:55:54.785174 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:54.785178 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:54.785191 (+ 13us) server_negotiation.cc:407] Connection header received
0811 18:55:54.785304 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:54.785306 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:54.785348 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:54.785420 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:54.785951 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:54.786510 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:54.787326 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:54.787570 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:54.787652 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:54.790223 (+ 2571us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.180.129:42583: BlockingRecv error: recv got EOF from 127.27.180.129:42583 (error 108)
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":75,"threads_started":1}
Aug 11 18:55:54 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.130@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:55:54 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.130@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:55:54.963418 13620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:54.957832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33277 (local address 127.27.180.130:55461)
0811 18:55:54.958110 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:54.958123 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:54.958233 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:54.958480 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:54.958484 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:54.958679 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:54.958777 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:54.958783 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:54.959626 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:54.959630 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:54.960249 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:54.960262 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:54.960744 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:54.960748 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:54.960832 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:54.961350 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:54.961362 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:54.963269 (+ 1907us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.180.190:33277: Server oryx/127.27.180.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":82,"threads_started":1}
W20260811 18:55:54.963532 13618 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:55:54.957916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.130:55461 (local address 127.27.180.190:33277)
0811 18:55:54.958073 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:54.958078 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:54.958186 (+ 108us) server_negotiation.cc:407] Connection header received
0811 18:55:54.958317 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:54.958320 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:54.958368 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:54.958433 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:54.958928 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:54.959523 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:54.960362 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:54.960638 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:54.960726 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:54.963420 (+ 2694us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.180.130:55461: BlockingRecv error: recv got EOF from 127.27.180.130:55461 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 11 18:55:55 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.131@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:55:55 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.131@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:55:55.158390 13621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:55.153233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33277 (local address 127.27.180.131:38809)
0811 18:55:55.153487 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:55.153499 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:55.153602 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:55.153809 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:55.153811 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:55.154009 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:55.154109 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:55.154115 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:55.154818 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:55.154821 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:55.155387 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:55.155395 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:55.155877 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:55.155881 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:55.155968 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:55.156396 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:55.156419 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:55.158218 (+ 1799us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.180.190:33277: Server oryx/127.27.180.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":69,"threads_started":1}
W20260811 18:55:55.158458 13618 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:55:55.153327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.131:38809 (local address 127.27.180.190:33277)
0811 18:55:55.153489 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:55.153493 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:55.153563 (+ 70us) server_negotiation.cc:407] Connection header received
0811 18:55:55.153670 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:55.153673 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:55.153718 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:55.153781 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:55.154194 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:55.154724 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:55.155527 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:55.155746 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:55.155833 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:55.158360 (+ 2527us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.180.131:38809: BlockingRecv error: recv got EOF from 127.27.180.131:38809 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 11 18:55:55 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.129@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:55:55 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.129@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:55:55.796715 13622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:55.790928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33277 (local address 127.27.180.129:60047)
0811 18:55:55.791261 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:55.791275 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:55.791388 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:55.791639 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:55.791643 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:55.791875 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:55.792013 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:55.792024 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:55.793109 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:55.793114 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:55.793702 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:55.793712 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:55.794105 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:55.794107 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:55.794231 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:55.794767 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:55.794793 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:55.796592 (+ 1799us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.180.190:33277: Server oryx/127.27.180.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":90,"threads_started":1}
W20260811 18:55:55.796836 13623 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:55:55.791028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.129:60047 (local address 127.27.180.190:33277)
0811 18:55:55.791260 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:55.791263 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:55.791344 (+ 81us) server_negotiation.cc:407] Connection header received
0811 18:55:55.791464 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:55.791466 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:55.791516 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:55.791601 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:55.792160 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:55.792972 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:55.793803 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:55.794027 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:55.794092 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:55.796725 (+ 2633us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.180.129:60047: BlockingRecv error: recv got EOF from 127.27.180.129:60047 (error 108)
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":56,"threads_started":1}
Aug 11 18:55:55 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.130@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:55:55 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.130@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:55:55.969305 13626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:55.964175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33277 (local address 127.27.180.130:54753)
0811 18:55:55.964490 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:55.964503 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:55.964602 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:55.964804 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:55.964806 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:55.964987 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:55.965117 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:55.965125 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:55.965814 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:55.965816 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:55.966388 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:55.966396 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:55.966770 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:55.966772 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:55.966833 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:55.967352 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:55.967364 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:55.969152 (+ 1788us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.180.190:33277: Server oryx/127.27.180.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":95,"threads_started":1}
W20260811 18:55:55.969375 13623 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:55:55.964262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.130:54753 (local address 127.27.180.190:33277)
0811 18:55:55.964399 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:55.964403 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:55.964563 (+ 160us) server_negotiation.cc:407] Connection header received
0811 18:55:55.964674 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:55.964676 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:55.964721 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:55.964777 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:55.965207 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:55.965728 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:55.966471 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:55.966684 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:55.966752 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:55.969278 (+ 2526us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.180.130:54753: BlockingRecv error: recv got EOF from 127.27.180.130:54753 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 11 18:55:56 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.131@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:55:56 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.131@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:55:56.167418 13628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:56.159148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33277 (local address 127.27.180.131:41703)
0811 18:55:56.159415 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:56.159428 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:56.159527 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:56.159762 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:56.159763 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:56.159978 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:56.160082 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:56.160089 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:56.160812 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:56.160814 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:56.161456 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:56.161467 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:56.161915 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:56.161918 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:56.161979 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:56.165113 (+ 3134us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:56.165124 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:56.167291 (+ 2167us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.180.190:33277: Server oryx/127.27.180.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":71,"threads_started":1}
W20260811 18:55:56.167514 13623 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:55:56.159251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.131:41703 (local address 127.27.180.190:33277)
0811 18:55:56.159402 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:56.159407 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:56.159517 (+ 110us) server_negotiation.cc:407] Connection header received
0811 18:55:56.159597 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:56.159600 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:56.159659 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:56.159740 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:56.160177 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:56.160722 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:56.161577 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:56.161814 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:56.161890 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:56.167406 (+ 5516us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.180.131:41703: BlockingRecv error: recv got EOF from 127.27.180.131:41703 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 11 18:55:56 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.129@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:55:56 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.129@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:55:56.802553 13629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:56.797488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33277 (local address 127.27.180.129:33631)
0811 18:55:56.797763 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:56.797777 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:56.797878 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:56.798085 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:56.798086 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:56.798285 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:56.798390 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:56.798396 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:56.799086 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:56.799088 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:56.799634 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:56.799640 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:56.800058 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:56.800062 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:56.800135 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:56.800607 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:56.800621 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:56.802432 (+ 1811us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.180.190:33277: Server oryx/127.27.180.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":72,"threads_started":1}
W20260811 18:55:56.802752 13630 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:55:56.797555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.129:33631 (local address 127.27.180.190:33277)
0811 18:55:56.797778 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:56.797782 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:56.797834 (+ 52us) server_negotiation.cc:407] Connection header received
0811 18:55:56.797942 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:56.797944 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:56.797987 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:56.798062 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:56.798479 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:56.799009 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:56.799716 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:56.799934 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:56.799999 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:56.802592 (+ 2593us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.180.129:33631: BlockingRecv error: recv got EOF from 127.27.180.129:33631 (error 108)
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":68,"threads_started":1}
Aug 11 18:55:56 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.130@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:55:56 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.130@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:55:56.975209 13635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:56.970118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33277 (local address 127.27.180.130:51651)
0811 18:55:56.970429 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:56.970442 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:56.970539 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:56.970748 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:56.970750 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:56.970935 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:56.971032 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:56.971037 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:56.971709 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:56.971711 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:56.972278 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:56.972285 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:56.972662 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:56.972665 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:56.972742 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:56.973230 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:56.973243 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:56.975077 (+ 1834us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.180.190:33277: Server oryx/127.27.180.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":131,"threads_started":1}
W20260811 18:55:56.975303 13630 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:55:56.970264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.130:51651 (local address 127.27.180.190:33277)
0811 18:55:56.970398 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:56.970401 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:56.970501 (+ 100us) server_negotiation.cc:407] Connection header received
0811 18:55:56.970606 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:56.970608 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:56.970650 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:56.970714 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:56.971113 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:56.971628 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:56.972362 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:56.972580 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:56.972651 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:56.975207 (+ 2556us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.180.130:51651: BlockingRecv error: recv got EOF from 127.27.180.130:51651 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 11 18:55:57 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.131@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:55:57 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.131@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:55:57.173974 13637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:57.168202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33277 (local address 127.27.180.131:58443)
0811 18:55:57.168513 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:57.168526 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:57.168624 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:57.168831 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:57.168833 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:57.169014 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:57.169158 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:57.169169 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:57.170162 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:57.170166 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:57.170816 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:57.170829 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:57.171308 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:57.171311 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:57.171387 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:57.171952 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:57.171965 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:57.173847 (+ 1882us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.180.190:33277: Server oryx/127.27.180.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":87,"threads_started":1}
W20260811 18:55:57.174086 13630 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:55:57.168296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.131:58443 (local address 127.27.180.190:33277)
0811 18:55:57.168451 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:57.168455 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:57.168586 (+ 131us) server_negotiation.cc:407] Connection header received
0811 18:55:57.168686 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:57.168689 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:57.168738 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:57.168809 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:57.169283 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:57.170039 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:57.170923 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:57.171151 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:57.171240 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:57.173983 (+ 2743us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.180.131:58443: BlockingRecv error: recv got EOF from 127.27.180.131:58443 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 11 18:55:57 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.129@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:55:57 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.129@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:55:57.808537 13639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:57.803379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33277 (local address 127.27.180.129:56079)
0811 18:55:57.803679 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:57.803693 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:57.803790 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:57.804010 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:57.804012 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:57.804196 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:57.804296 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:57.804302 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:57.805042 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:57.805044 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:57.805620 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:57.805627 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:57.806092 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:57.806094 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:57.806151 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:57.806616 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:57.806628 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:57.808422 (+ 1794us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.180.190:33277: Server oryx/127.27.180.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":87,"threads_started":1}
W20260811 18:55:57.808748 13640 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:55:57.803457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.129:56079 (local address 127.27.180.190:33277)
0811 18:55:57.803784 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:57.803788 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:57.803802 (+ 14us) server_negotiation.cc:407] Connection header received
0811 18:55:57.803860 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:57.803862 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:57.803907 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:57.803989 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:57.804396 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:57.804961 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:57.805740 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:57.805997 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:57.806089 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:57.808613 (+ 2524us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.180.129:56079: BlockingRecv error: recv got EOF from 127.27.180.129:56079 (error 108)
Metrics: {"server-negotiator.queue_time_us":253,"thread_start_us":80,"threads_started":1}
Aug 11 18:55:57 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.130@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:55:57 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.130@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:55:57.981544 13641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:57.976028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33277 (local address 127.27.180.130:34647)
0811 18:55:57.976290 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:57.976304 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:57.976407 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:57.976621 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:57.976623 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:57.976811 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:57.976927 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:57.976933 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:57.977727 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:57.977730 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:57.978303 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:57.978312 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:57.978742 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:57.978746 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:57.978837 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:57.979408 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:57.979425 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:57.981394 (+ 1969us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.180.190:33277: Server oryx/127.27.180.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":73,"threads_started":1}
W20260811 18:55:57.981652 13640 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:55:57.976125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.130:34647 (local address 127.27.180.190:33277)
0811 18:55:57.976286 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:57.976290 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:57.976371 (+ 81us) server_negotiation.cc:407] Connection header received
0811 18:55:57.976477 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:57.976479 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:57.976524 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:57.976590 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:57.977022 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:57.977618 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:57.978393 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:57.978617 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:57.978705 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:57.981552 (+ 2847us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.180.130:34647: BlockingRecv error: recv got EOF from 127.27.180.130:34647 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 11 18:55:58 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.131@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:55:58 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.131@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:55:58.179754 13642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:58.174714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33277 (local address 127.27.180.131:36943)
0811 18:55:58.174968 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:58.174981 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:58.175083 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:58.175283 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:58.175285 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:58.175479 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:58.175628 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:58.175640 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:58.176371 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:58.176374 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:58.176993 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:58.177004 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:58.177426 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:58.177428 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:58.177501 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:58.177928 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:58.177939 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:58.179617 (+ 1678us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.180.190:33277: Server oryx/127.27.180.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":86,"threads_started":1}
W20260811 18:55:58.179859 13640 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:55:58.174764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.131:36943 (local address 127.27.180.190:33277)
0811 18:55:58.174912 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:58.174917 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:58.175031 (+ 114us) server_negotiation.cc:407] Connection header received
0811 18:55:58.175145 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:58.175147 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:58.175192 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:58.175255 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:58.175734 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:58.176268 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:58.177109 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:58.177337 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:58.177425 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:58.179751 (+ 2326us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.180.131:36943: BlockingRecv error: recv got EOF from 127.27.180.131:36943 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 11 18:55:58 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.129@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:55:58 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.129@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:55:58.814582 13643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:58.809538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33277 (local address 127.27.180.129:56003)
0811 18:55:58.809881 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:58.809895 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:58.810005 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:58.810217 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:58.810219 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:58.810410 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:58.810506 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:58.810512 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:58.811197 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:58.811199 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:58.811736 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:58.811742 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:58.812185 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:58.812187 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:58.812241 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:58.812716 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:58.812729 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:58.814469 (+ 1740us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.180.190:33277: Server oryx/127.27.180.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":81,"threads_started":1}
W20260811 18:55:58.814736 13646 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:55:58.809592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.129:56003 (local address 127.27.180.190:33277)
0811 18:55:58.809841 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:58.809844 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:58.809972 (+ 128us) server_negotiation.cc:407] Connection header received
0811 18:55:58.810083 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:58.810085 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:58.810128 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:58.810217 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:58.810588 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:58.811117 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:58.811851 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:58.812091 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:58.812171 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:58.814603 (+ 2432us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.180.129:56003: BlockingRecv error: recv got EOF from 127.27.180.129:56003 (error 108)
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":58,"threads_started":1}
Aug 11 18:55:58 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.130@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 18:55:58 dist-test-slave-2j7r krb5kdc[11605](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.27.180.130@KRBTEST.COM for oryx/127.27.180.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 18:55:58.987820 13647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:58.982552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33277 (local address 127.27.180.130:47473)
0811 18:55:58.982861 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:58.982876 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:58.982980 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:58.983252 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:58.983255 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:58.983455 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:58.983560 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:58.983566 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:58.984329 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:58.984332 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:58.984917 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:58.984927 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:58.985352 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:58.985355 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:58.985423 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:58.985875 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:55:58.985886 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:58.987699 (+ 1813us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.27.180.190:33277: Server oryx/127.27.180.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":91,"threads_started":1}
W20260811 18:55:58.987944 13646 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:55:58.982616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.130:47473 (local address 127.27.180.190:33277)
0811 18:55:58.982744 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:58.982747 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:58.982941 (+ 194us) server_negotiation.cc:407] Connection header received
0811 18:55:58.983054 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:58.983057 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:58.983101 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:58.983168 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:58.983669 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:58.984199 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:58.985014 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:58.985253 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:58.985325 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:58.987839 (+ 2514us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.27.180.130:47473: BlockingRecv error: recv got EOF from 127.27.180.130:47473 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 18:55:59.068526 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 12747
I20260811 18:55:59.085405 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 13035
I20260811 18:55:59.094594 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 13280
I20260811 18:55:59.103942 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 12613
2026-08-11T18:55:59Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18113 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-11T18:55:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:55:59Z Disabled control of system clock
I20260811 18:55:59.140645 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474496157106-28370-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474496157106-28370-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474496157106-28370-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.190:33321
--webserver_interface=127.27.180.190
--webserver_port=0
--builtin_ntp_servers=127.27.180.148:38675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.180.190:33321
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260811 18:55:59.249145 13653 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:55:59.249397 13653 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:55:59.249428 13653 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:55:59.253280 13653 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:55:59.253327 13653 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:55:59.253341 13653 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:55:59.253356 13653 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:55:59.253369 13653 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:55:59.259158 13653 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:38675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474496157106-28370-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474496157106-28370-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.180.190:33321
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.190:33321
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.180.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13653
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474496157106-28370-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:55:59.260736 13653 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:55:59.262689 13653 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 18:55:59.271752 13653 server_base.cc:1034] running on GCE node
W20260811 18:55:59.271695 13662 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:59.271718 13661 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:59.271692 13665 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:55:59.272563 13653 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:55:59.273715 13653 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:55:59.274864 13653 hybrid_clock.cc:648] HybridClock initialized: now 1786474559274843 us; error 38 us; skew 500 ppm
I20260811 18:55:59.276683 13653 webserver.cc:460] Webserver started at http://127.27.180.190:45819/ using document root <none> and password file <none>
I20260811 18:55:59.277274 13653 fs_manager.cc:359] Metadata directory not provided
I20260811 18:55:59.277338 13653 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:55:59.277516 13653 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:55:59.279526 13653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474496157106-28370-0/minicluster-data/master-0/data/instance:
uuid: "c6349398c5ca4988a9e926124d67b93e"
format_stamp: "Formatted at 2026-08-11 18:55:59 on dist-test-slave-2j7r"
server_key: "3aa60faf221b5166f15a66426d6399aa"
server_key_iv: "5c3f46fb127310857e42d77f6de3bcdf"
server_key_version: "encryptionkey@0"
I20260811 18:55:59.280052 13653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474496157106-28370-0/minicluster-data/master-0/wal/instance:
uuid: "c6349398c5ca4988a9e926124d67b93e"
format_stamp: "Formatted at 2026-08-11 18:55:59 on dist-test-slave-2j7r"
server_key: "3aa60faf221b5166f15a66426d6399aa"
server_key_iv: "5c3f46fb127310857e42d77f6de3bcdf"
server_key_version: "encryptionkey@0"
I20260811 18:55:59.283702 13653 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260811 18:55:59.286041 13677 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:59.287264 13653 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 18:55:59.287379 13653 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474496157106-28370-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474496157106-28370-0/minicluster-data/master-0/wal
uuid: "c6349398c5ca4988a9e926124d67b93e"
format_stamp: "Formatted at 2026-08-11 18:55:59 on dist-test-slave-2j7r"
server_key: "3aa60faf221b5166f15a66426d6399aa"
server_key_iv: "5c3f46fb127310857e42d77f6de3bcdf"
server_key_version: "encryptionkey@0"
I20260811 18:55:59.287485 13653 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474496157106-28370-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474496157106-28370-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474496157106-28370-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:55:59.298002 13653 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:55:59.298628 13653 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:55:59.298794 13653 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:55:59.305830 13653 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.190:33321
I20260811 18:55:59.305886 13766 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.190:33321 every 8 connection(s)
I20260811 18:55:59.306826 13653 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
I20260811 18:55:59.310137 13768 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:55:59.315959 13768 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6349398c5ca4988a9e926124d67b93e: Bootstrap starting.
I20260811 18:55:59.316349 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 13653
I20260811 18:55:59.316478 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474496157106-28370-0/minicluster-data/master-0/wal/instance
I20260811 18:55:59.316846 28370 external_mini_cluster.cc:1428] Setting key 108c258508317b4cdb704c684749b380
I20260811 18:55:59.318830 13768 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c6349398c5ca4988a9e926124d67b93e: Neither blocks nor log segments found. Creating new log.
I20260811 18:55:59.319579 13768 log.cc:824] T 00000000000000000000000000000000 P c6349398c5ca4988a9e926124d67b93e: Log is configured to *not* fsync() on all Append() calls
I20260811 18:55:59.321681 13768 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6349398c5ca4988a9e926124d67b93e: No bootstrap required, opened a new log
I20260811 18:55:59.325155 13768 raft_consensus.cc:348] T 00000000000000000000000000000000 P c6349398c5ca4988a9e926124d67b93e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6349398c5ca4988a9e926124d67b93e" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 33321 } }
I20260811 18:55:59.325376 13768 raft_consensus.cc:374] T 00000000000000000000000000000000 P c6349398c5ca4988a9e926124d67b93e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:55:59.325434 13768 raft_consensus.cc:729] T 00000000000000000000000000000000 P c6349398c5ca4988a9e926124d67b93e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6349398c5ca4988a9e926124d67b93e, State: Initialized, Role: FOLLOWER
I20260811 18:55:59.325857 13772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:59.318274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40528 (local address 127.27.180.190:33321)
0811 18:55:59.318872 (+ 598us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:59.318890 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:59.318938 (+ 48us) server_negotiation.cc:407] Connection header received
0811 18:55:59.319578 (+ 640us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:59.319595 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:59.320040 (+ 445us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:59.320410 (+ 370us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:59.320750 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:59.321749 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:59.322611 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:59.323071 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:59.323223 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:59.323997 (+ 774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:59.324001 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:59.324013 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:55:59.324388 (+ 375us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:59.324456 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:55:59.324476 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:59.324560 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:59.324703 (+ 143us) server_negotiation.cc:299] Negotiation successful
0811 18:55:59.324959 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":257,"thread_start_us":104,"threads_started":1}
I20260811 18:55:59.325837 13768 consensus_queue.cc:260] T 00000000000000000000000000000000 P c6349398c5ca4988a9e926124d67b93e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6349398c5ca4988a9e926124d67b93e" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 33321 } }
I20260811 18:55:59.325966 13768 raft_consensus.cc:388] T 00000000000000000000000000000000 P c6349398c5ca4988a9e926124d67b93e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:55:59.326020 13768 raft_consensus.cc:482] T 00000000000000000000000000000000 P c6349398c5ca4988a9e926124d67b93e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:55:59.326144 13768 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c6349398c5ca4988a9e926124d67b93e [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:55:59.327090 13768 raft_consensus.cc:504] T 00000000000000000000000000000000 P c6349398c5ca4988a9e926124d67b93e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6349398c5ca4988a9e926124d67b93e" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 33321 } }
I20260811 18:55:59.327462 13768 leader_election.cc:302] T 00000000000000000000000000000000 P c6349398c5ca4988a9e926124d67b93e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c6349398c5ca4988a9e926124d67b93e; no voters:
I20260811 18:55:59.327791 13768 leader_election.cc:288] T 00000000000000000000000000000000 P c6349398c5ca4988a9e926124d67b93e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:55:59.327878 13774 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c6349398c5ca4988a9e926124d67b93e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:55:59.328159 13774 raft_consensus.cc:686] T 00000000000000000000000000000000 P c6349398c5ca4988a9e926124d67b93e [term 1 LEADER]: Becoming Leader. State: Replica: c6349398c5ca4988a9e926124d67b93e, State: Running, Role: LEADER
I20260811 18:55:59.328441 13774 consensus_queue.cc:237] T 00000000000000000000000000000000 P c6349398c5ca4988a9e926124d67b93e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6349398c5ca4988a9e926124d67b93e" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 33321 } }
I20260811 18:55:59.329062 13768 sys_catalog.cc:564] T 00000000000000000000000000000000 P c6349398c5ca4988a9e926124d67b93e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:55:59.330060 13778 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6349398c5ca4988a9e926124d67b93e [sys.catalog]: SysCatalogTable state changed. Reason: New leader c6349398c5ca4988a9e926124d67b93e. Latest consensus state: current_term: 1 leader_uuid: "c6349398c5ca4988a9e926124d67b93e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6349398c5ca4988a9e926124d67b93e" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 33321 } } }
I20260811 18:55:59.330060 13776 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6349398c5ca4988a9e926124d67b93e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c6349398c5ca4988a9e926124d67b93e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6349398c5ca4988a9e926124d67b93e" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 33321 } } }
I20260811 18:55:59.330233 13778 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6349398c5ca4988a9e926124d67b93e [sys.catalog]: This master's current role is: LEADER
I20260811 18:55:59.330268 13776 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6349398c5ca4988a9e926124d67b93e [sys.catalog]: This master's current role is: LEADER
I20260811 18:55:59.330593 13787 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:55:59.333739 13787 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:55:59.339309 13787 catalog_manager.cc:1347] Generated new cluster ID: 51a5e00b141e4f44a3f331d8d8a7e9a0
I20260811 18:55:59.339361 13787 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:55:59.351240 13787 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:55:59.352141 13787 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:55:59.356285 13787 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c6349398c5ca4988a9e926124d67b93e: Generated new TSK 0
I20260811 18:55:59.356843 13787 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:55:59.363534 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474496157106-28370-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474496157106-28370-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474496157106-28370-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474496157106-28370-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.180.129:0
--local_ip_for_outbound_sockets=127.27.180.129
--webserver_interface=127.27.180.129
--webserver_port=0
--tserver_master_addrs=127.27.180.190:33321
--builtin_ntp_servers=127.27.180.148:38675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260811 18:55:59.491482 13805 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:55:59.491719 13805 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:55:59.491756 13805 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:55:59.495677 13805 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:55:59.495726 13805 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:55:59.495776 13805 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.180.129
I20260811 18:55:59.500389 13805 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:38675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474496157106-28370-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474496157106-28370-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.180.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474496157106-28370-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.180.129
--webserver_port=0
--tserver_master_addrs=127.27.180.190:33321
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13805
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.180.129
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474496157106-28370-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:55:59.501456 13805 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:55:59.502796 13805 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:55:59.512090 13813 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:59.512086 13816 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:59.512105 13814 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:55:59.512668 13805 server_base.cc:1034] running on GCE node
I20260811 18:55:59.513046 13805 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:55:59.513722 13805 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:55:59.514914 13805 hybrid_clock.cc:648] HybridClock initialized: now 1786474559514876 us; error 40 us; skew 500 ppm
I20260811 18:55:59.516822 13805 webserver.cc:460] Webserver started at http://127.27.180.129:44005/ using document root <none> and password file <none>
I20260811 18:55:59.517431 13805 fs_manager.cc:359] Metadata directory not provided
I20260811 18:55:59.517501 13805 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:55:59.517729 13805 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:55:59.519764 13805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474496157106-28370-0/minicluster-data/ts-0/data/instance:
uuid: "358f134d45974ef0a1c9ffebe35ae4a7"
format_stamp: "Formatted at 2026-08-11 18:55:59 on dist-test-slave-2j7r"
server_key: "f4ac1d59595a0bb22cf9f70e04c6e939"
server_key_iv: "d83b57133baae0103c2967038de4a7f2"
server_key_version: "encryptionkey@0"
I20260811 18:55:59.520354 13805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474496157106-28370-0/minicluster-data/ts-0/wal/instance:
uuid: "358f134d45974ef0a1c9ffebe35ae4a7"
format_stamp: "Formatted at 2026-08-11 18:55:59 on dist-test-slave-2j7r"
server_key: "f4ac1d59595a0bb22cf9f70e04c6e939"
server_key_iv: "d83b57133baae0103c2967038de4a7f2"
server_key_version: "encryptionkey@0"
I20260811 18:55:59.524056 13805 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260811 18:55:59.526531 13827 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:59.527674 13805 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260811 18:55:59.527773 13805 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474496157106-28370-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474496157106-28370-0/minicluster-data/ts-0/wal
uuid: "358f134d45974ef0a1c9ffebe35ae4a7"
format_stamp: "Formatted at 2026-08-11 18:55:59 on dist-test-slave-2j7r"
server_key: "f4ac1d59595a0bb22cf9f70e04c6e939"
server_key_iv: "d83b57133baae0103c2967038de4a7f2"
server_key_version: "encryptionkey@0"
I20260811 18:55:59.527873 13805 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474496157106-28370-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474496157106-28370-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474496157106-28370-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:55:59.550158 13805 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:55:59.550848 13805 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:55:59.551016 13805 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:55:59.551580 13805 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:55:59.552506 13805 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:55:59.552554 13805 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:59.552606 13805 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:55:59.552634 13805 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:59.561044 13805 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.129:43519
I20260811 18:55:59.561165 14039 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.129:43519 every 8 connection(s)
I20260811 18:55:59.562039 13805 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474496157106-28370-0/minicluster-data/ts-0/data/info.pb
I20260811 18:55:59.570848 13772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:59.563737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.129:55825 (local address 127.27.180.190:33321)
0811 18:55:59.563864 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:59.563867 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:59.564885 (+ 1018us) server_negotiation.cc:407] Connection header received
0811 18:55:59.566324 (+ 1439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:59.566327 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:59.566386 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:59.566465 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:59.567355 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:59.567892 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:59.569051 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:59.569324 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:59.569428 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:59.570393 (+ 965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:59.570396 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:59.570398 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:55:59.570418 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:59.570437 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:55:59.570445 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:59.570504 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:59.570639 (+ 135us) server_negotiation.cc:299] Negotiation successful
0811 18:55:59.570704 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 18:55:59.571264 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 13805
I20260811 18:55:59.571379 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474496157106-28370-0/minicluster-data/ts-0/wal/instance
I20260811 18:55:59.571730 28370 external_mini_cluster.cc:1428] Setting key de8637737370219806d3dd242eecc313
I20260811 18:55:59.571832 14044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:59.564078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33321 (local address 127.27.180.129:55825)
0811 18:55:59.564739 (+ 661us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:59.564777 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:59.566110 (+ 1333us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:59.566663 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:59.566670 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:59.566750 (+ 80us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:59.567210 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:59.567223 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:59.568023 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:59.568026 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:59.568927 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:59.568937 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:59.569473 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:59.569478 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:59.569632 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:59.570147 (+ 515us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:55:59.570149 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:59.570186 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:55:59.570247 (+ 61us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:55:59.570513 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:55:59.570523 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:55:59.570529 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 18:55:59.570631 (+ 102us) client_negotiation.cc:240] Negotiation successful
0811 18:55:59.570888 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":147,"threads_started":1}
I20260811 18:55:59.573145 14040 heartbeater.cc:347] Connected to a master server at 127.27.180.190:33321
I20260811 18:55:59.573369 14040 heartbeater.cc:464] Registering TS with master...
I20260811 18:55:59.573858 14040 heartbeater.cc:511] Master 127.27.180.190:33321 requested a full tablet report, sending...
I20260811 18:55:59.573930 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474496157106-28370-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474496157106-28370-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474496157106-28370-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474496157106-28370-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.180.130:0
--local_ip_for_outbound_sockets=127.27.180.130
--webserver_interface=127.27.180.130
--webserver_port=0
--tserver_master_addrs=127.27.180.190:33321
--builtin_ntp_servers=127.27.180.148:38675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260811 18:55:59.575392 13706 ts_manager.cc:194] Registered new tserver with Master: 358f134d45974ef0a1c9ffebe35ae4a7 (127.27.180.129:43519)
I20260811 18:55:59.576572 13706 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.27.180.129:55825
W20260811 18:55:59.686738 14046 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:55:59.686970 14046 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:55:59.687005 14046 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:55:59.690930 14046 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:55:59.690974 14046 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:55:59.691032 14046 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.180.130
I20260811 18:55:59.695636 14046 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:38675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474496157106-28370-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474496157106-28370-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.180.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474496157106-28370-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.180.130
--webserver_port=0
--tserver_master_addrs=127.27.180.190:33321
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14046
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.180.130
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474496157106-28370-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:55:59.696538 14046 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:55:59.697810 14046 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:55:59.707134 14064 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:59.707121 14057 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:55:59.707228 14046 server_base.cc:1034] running on GCE node
W20260811 18:55:59.707134 14059 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:55:59.707836 14046 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:55:59.708438 14046 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:55:59.709605 14046 hybrid_clock.cc:648] HybridClock initialized: now 1786474559709577 us; error 43 us; skew 500 ppm
I20260811 18:55:59.711300 14046 webserver.cc:460] Webserver started at http://127.27.180.130:33039/ using document root <none> and password file <none>
I20260811 18:55:59.711839 14046 fs_manager.cc:359] Metadata directory not provided
I20260811 18:55:59.711906 14046 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:55:59.712107 14046 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:55:59.714130 14046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474496157106-28370-0/minicluster-data/ts-1/data/instance:
uuid: "db9245657a4944e4b97f4e78fdc2dcc8"
format_stamp: "Formatted at 2026-08-11 18:55:59 on dist-test-slave-2j7r"
server_key: "1916ea9382be1b53c0282f0d2418a0bb"
server_key_iv: "4993f0eed17d90e5757fb22dc9198c5a"
server_key_version: "encryptionkey@0"
I20260811 18:55:59.714658 14046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474496157106-28370-0/minicluster-data/ts-1/wal/instance:
uuid: "db9245657a4944e4b97f4e78fdc2dcc8"
format_stamp: "Formatted at 2026-08-11 18:55:59 on dist-test-slave-2j7r"
server_key: "1916ea9382be1b53c0282f0d2418a0bb"
server_key_iv: "4993f0eed17d90e5757fb22dc9198c5a"
server_key_version: "encryptionkey@0"
I20260811 18:55:59.718101 14046 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.004s
I20260811 18:55:59.720496 14076 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:59.721709 14046 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 18:55:59.721817 14046 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474496157106-28370-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474496157106-28370-0/minicluster-data/ts-1/wal
uuid: "db9245657a4944e4b97f4e78fdc2dcc8"
format_stamp: "Formatted at 2026-08-11 18:55:59 on dist-test-slave-2j7r"
server_key: "1916ea9382be1b53c0282f0d2418a0bb"
server_key_iv: "4993f0eed17d90e5757fb22dc9198c5a"
server_key_version: "encryptionkey@0"
I20260811 18:55:59.721909 14046 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474496157106-28370-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474496157106-28370-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474496157106-28370-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:55:59.734431 14046 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:55:59.735034 14046 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:55:59.735195 14046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:55:59.735865 14046 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:55:59.736835 14046 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:55:59.736881 14046 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:59.736922 14046 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:55:59.736950 14046 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:59.745251 14046 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.130:38483
I20260811 18:55:59.745266 14264 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.130:38483 every 8 connection(s)
I20260811 18:55:59.746119 14046 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474496157106-28370-0/minicluster-data/ts-1/data/info.pb
I20260811 18:55:59.752139 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 14046
I20260811 18:55:59.752241 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474496157106-28370-0/minicluster-data/ts-1/wal/instance
I20260811 18:55:59.752517 28370 external_mini_cluster.cc:1428] Setting key 333cc0b9a8943179ea0205270e328a91
I20260811 18:55:59.754523 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474496157106-28370-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474496157106-28370-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474496157106-28370-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474496157106-28370-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.180.131:0
--local_ip_for_outbound_sockets=127.27.180.131
--webserver_interface=127.27.180.131
--webserver_port=0
--tserver_master_addrs=127.27.180.190:33321
--builtin_ntp_servers=127.27.180.148:38675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260811 18:55:59.755227 13772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:59.747876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.130:46137 (local address 127.27.180.190:33321)
0811 18:55:59.748027 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:59.748031 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:59.749043 (+ 1012us) server_negotiation.cc:407] Connection header received
0811 18:55:59.750435 (+ 1392us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:59.750439 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:59.750496 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:59.750566 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:59.751439 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:59.751999 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:59.753296 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:59.753556 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:59.753636 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:59.754748 (+ 1112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:59.754751 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:59.754753 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:55:59.754765 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:59.754781 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:55:59.754789 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:59.754824 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:59.755019 (+ 195us) server_negotiation.cc:299] Negotiation successful
0811 18:55:59.755081 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":64}
I20260811 18:55:59.755967 14268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:59.748244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33321 (local address 127.27.180.130:46137)
0811 18:55:59.748888 (+ 644us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:59.748936 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:59.750210 (+ 1274us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:59.750738 (+ 528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:59.750745 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:59.750826 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:59.751293 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:59.751306 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:59.752156 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:59.752159 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:59.753175 (+ 1016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:59.753188 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:59.753712 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:59.753729 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:59.753873 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:59.754451 (+ 578us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:55:59.754452 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:59.754493 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:55:59.754559 (+ 66us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:55:59.754877 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:55:59.754885 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:55:59.754888 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 18:55:59.754973 (+ 85us) client_negotiation.cc:240] Negotiation successful
0811 18:55:59.755187 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":128,"threads_started":1}
I20260811 18:55:59.757055 14266 heartbeater.cc:347] Connected to a master server at 127.27.180.190:33321
I20260811 18:55:59.757325 14266 heartbeater.cc:464] Registering TS with master...
I20260811 18:55:59.757803 14266 heartbeater.cc:511] Master 127.27.180.190:33321 requested a full tablet report, sending...
I20260811 18:55:59.758853 13706 ts_manager.cc:194] Registered new tserver with Master: db9245657a4944e4b97f4e78fdc2dcc8 (127.27.180.130:38483)
I20260811 18:55:59.759402 13706 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.27.180.130:46137
W20260811 18:55:59.865204 14270 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:55:59.865442 14270 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:55:59.865491 14270 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:55:59.869407 14270 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:55:59.869452 14270 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:55:59.869519 14270 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.180.131
I20260811 18:55:59.874151 14270 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:38675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474496157106-28370-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474496157106-28370-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.180.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474496157106-28370-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.180.131
--webserver_port=0
--tserver_master_addrs=127.27.180.190:33321
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14270
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.180.131
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474496157106-28370-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:55:59.875104 14270 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:55:59.876372 14270 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:55:59.885483 14279 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:55:59.885473 14280 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:55:59.885551 14270 server_base.cc:1034] running on GCE node
W20260811 18:55:59.885474 14282 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:55:59.886070 14270 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:55:59.886655 14270 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:55:59.887811 14270 hybrid_clock.cc:648] HybridClock initialized: now 1786474559887793 us; error 28 us; skew 500 ppm
I20260811 18:55:59.889554 14270 webserver.cc:460] Webserver started at http://127.27.180.131:41241/ using document root <none> and password file <none>
I20260811 18:55:59.890077 14270 fs_manager.cc:359] Metadata directory not provided
I20260811 18:55:59.890143 14270 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:55:59.890321 14270 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:55:59.892364 14270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474496157106-28370-0/minicluster-data/ts-2/data/instance:
uuid: "687266e7d1e24396aae1045332ba95c7"
format_stamp: "Formatted at 2026-08-11 18:55:59 on dist-test-slave-2j7r"
server_key: "f7b8513cb7530e76b8cce1896e1d90aa"
server_key_iv: "b86920b21e97fbc375e65889945a7647"
server_key_version: "encryptionkey@0"
I20260811 18:55:59.892879 14270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474496157106-28370-0/minicluster-data/ts-2/wal/instance:
uuid: "687266e7d1e24396aae1045332ba95c7"
format_stamp: "Formatted at 2026-08-11 18:55:59 on dist-test-slave-2j7r"
server_key: "f7b8513cb7530e76b8cce1896e1d90aa"
server_key_iv: "b86920b21e97fbc375e65889945a7647"
server_key_version: "encryptionkey@0"
I20260811 18:55:59.896251 14270 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260811 18:55:59.898573 14292 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:59.899699 14270 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 18:55:59.899803 14270 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474496157106-28370-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474496157106-28370-0/minicluster-data/ts-2/wal
uuid: "687266e7d1e24396aae1045332ba95c7"
format_stamp: "Formatted at 2026-08-11 18:55:59 on dist-test-slave-2j7r"
server_key: "f7b8513cb7530e76b8cce1896e1d90aa"
server_key_iv: "b86920b21e97fbc375e65889945a7647"
server_key_version: "encryptionkey@0"
I20260811 18:55:59.899891 14270 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474496157106-28370-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474496157106-28370-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474496157106-28370-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:55:59.914813 14270 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:55:59.915403 14270 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:55:59.915586 14270 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:55:59.916173 14270 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:55:59.917042 14270 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:55:59.917109 14270 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:59.917165 14270 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:55:59.917194 14270 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:55:59.925377 14270 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.131:42759
I20260811 18:55:59.925405 14503 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.131:42759 every 8 connection(s)
I20260811 18:55:59.926223 14270 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474496157106-28370-0/minicluster-data/ts-2/data/info.pb
I20260811 18:55:59.932217 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 14270
I20260811 18:55:59.932319 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786474496157106-28370-0/minicluster-data/ts-2/wal/instance
I20260811 18:55:59.932595 28370 external_mini_cluster.cc:1428] Setting key dd927b169d79245c92e6cba34437ba80
I20260811 18:55:59.935766 13772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:59.927932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.131:37471 (local address 127.27.180.190:33321)
0811 18:55:59.928096 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:59.928100 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:59.929128 (+ 1028us) server_negotiation.cc:407] Connection header received
0811 18:55:59.930651 (+ 1523us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:59.930655 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:59.930728 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:59.930808 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:59.931812 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:59.932487 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:59.933878 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:55:59.934167 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:55:59.934278 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:59.935323 (+ 1045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:55:59.935345 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:55:59.935348 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:55:59.935360 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:55:59.935379 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:55:59.935388 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:55:59.935454 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 18:55:59.935603 (+ 149us) server_negotiation.cc:299] Negotiation successful
0811 18:55:59.935653 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 18:55:59.936630 14508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:55:59.928301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33321 (local address 127.27.180.131:37471)
0811 18:55:59.928936 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0811 18:55:59.928976 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 18:55:59.930428 (+ 1452us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:55:59.931024 (+ 596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:55:59.931031 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:55:59.931113 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:55:59.931656 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:59.931673 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:59.932620 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:59.932624 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:59.933759 (+ 1135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:55:59.933771 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:55:59.934329 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:55:59.934333 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:55:59.934476 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:55:59.935049 (+ 573us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:55:59.935051 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:55:59.935094 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:55:59.935157 (+ 63us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:55:59.935478 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:55:59.935487 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:55:59.935493 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 18:55:59.935596 (+ 103us) client_negotiation.cc:240] Negotiation successful
0811 18:55:59.935817 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":108,"threads_started":1}
I20260811 18:55:59.937686 14505 heartbeater.cc:347] Connected to a master server at 127.27.180.190:33321
I20260811 18:55:59.937901 14505 heartbeater.cc:464] Registering TS with master...
I20260811 18:55:59.938416 14505 heartbeater.cc:511] Master 127.27.180.190:33321 requested a full tablet report, sending...
I20260811 18:55:59.939353 13706 ts_manager.cc:194] Registered new tserver with Master: 687266e7d1e24396aae1045332ba95c7 (127.27.180.131:42759)
I20260811 18:55:59.939885 13706 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.27.180.131:37471
I20260811 18:55:59.946369 28370 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 18:55:59.949143 13772 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:55:59.948240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40544 (local address 127.27.180.190:33321)
0811 18:55:59.948354 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0811 18:55:59.948359 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:55:59.948541 (+ 182us) server_negotiation.cc:407] Connection header received
0811 18:55:59.948660 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:55:59.948663 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:55:59.948701 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:55:59.948761 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:55:59.949019 (+ 258us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40544: BlockingRecv error: recv got EOF from 127.0.0.1:40544 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 18:55:59.949468 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 13805
I20260811 18:55:59.976078 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 14046
I20260811 18:56:00.001873 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 14270
I20260811 18:56:00.010885 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 13653
2026-08-11T18:56:00Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (910 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-11T18:56:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:56:00Z Disabled control of system clock
I20260811 18:56:00.051918 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474496157106-28370-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474496157106-28370-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474496157106-28370-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.190:36091
--webserver_interface=127.27.180.190
--webserver_port=0
--builtin_ntp_servers=127.27.180.148:43007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.180.190:36091
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 18:56:00.163419 14524 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:56:00.163702 14524 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:56:00.163746 14524 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:56:00.167807 14524 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:56:00.167865 14524 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:56:00.167881 14524 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:56:00.167917 14524 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:56:00.167949 14524 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:56:00.172640 14524 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:43007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474496157106-28370-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474496157106-28370-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.180.190:36091
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.190:36091
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.180.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14524
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474496157106-28370-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:56:00.173763 14524 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:56:00.175139 14524 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:56:00.183454 14531 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:56:00.183471 14532 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:56:00.183471 14534 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:56:00.183817 14524 server_base.cc:1034] running on GCE node
I20260811 18:56:00.184347 14524 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:56:00.185446 14524 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:56:00.186601 14524 hybrid_clock.cc:648] HybridClock initialized: now 1786474560186586 us; error 27 us; skew 500 ppm
I20260811 18:56:00.188319 14524 webserver.cc:460] Webserver started at http://127.27.180.190:33465/ using document root <none> and password file <none>
I20260811 18:56:00.188845 14524 fs_manager.cc:359] Metadata directory not provided
I20260811 18:56:00.188920 14524 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:56:00.189165 14524 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:56:00.191318 14524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474496157106-28370-0/minicluster-data/master-0/data/instance:
uuid: "b8c228182f62487880d38e1203f9bfdd"
format_stamp: "Formatted at 2026-08-11 18:56:00 on dist-test-slave-2j7r"
server_key: "93641def0b3833e536e0a636f8a4405b"
server_key_iv: "d79a08d48e07888e0b05ff776d0a67f0"
server_key_version: "encryptionkey@0"
I20260811 18:56:00.191887 14524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474496157106-28370-0/minicluster-data/master-0/wal/instance:
uuid: "b8c228182f62487880d38e1203f9bfdd"
format_stamp: "Formatted at 2026-08-11 18:56:00 on dist-test-slave-2j7r"
server_key: "93641def0b3833e536e0a636f8a4405b"
server_key_iv: "d79a08d48e07888e0b05ff776d0a67f0"
server_key_version: "encryptionkey@0"
I20260811 18:56:00.195490 14524 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 18:56:00.197885 14546 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:56:00.199111 14524 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 18:56:00.199239 14524 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474496157106-28370-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474496157106-28370-0/minicluster-data/master-0/wal
uuid: "b8c228182f62487880d38e1203f9bfdd"
format_stamp: "Formatted at 2026-08-11 18:56:00 on dist-test-slave-2j7r"
server_key: "93641def0b3833e536e0a636f8a4405b"
server_key_iv: "d79a08d48e07888e0b05ff776d0a67f0"
server_key_version: "encryptionkey@0"
I20260811 18:56:00.199339 14524 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474496157106-28370-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474496157106-28370-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474496157106-28370-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:56:00.203763 14524 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:56:00.204340 14524 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:56:00.204504 14524 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:56:00.211287 14524 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.190:36091
I20260811 18:56:00.211305 14647 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.190:36091 every 8 connection(s)
I20260811 18:56:00.212153 14524 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
I20260811 18:56:00.215389 14652 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:56:00.217468 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 14524
I20260811 18:56:00.217588 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474496157106-28370-0/minicluster-data/master-0/wal/instance
I20260811 18:56:00.217904 28370 external_mini_cluster.cc:1428] Setting key b94e37c5211219cf1cca8c1cd28e6a71
I20260811 18:56:00.221437 14652 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8c228182f62487880d38e1203f9bfdd: Bootstrap starting.
I20260811 18:56:00.223522 14655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:56:00.219305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56422 (local address 127.27.180.190:36091)
0811 18:56:00.219931 (+ 626us) server_negotiation.cc:206] Beginning negotiation
0811 18:56:00.219946 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 18:56:00.219981 (+ 35us) server_negotiation.cc:407] Connection header received
0811 18:56:00.220634 (+ 653us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:56:00.220650 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:56:00.220999 (+ 349us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:56:00.221334 (+ 335us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:56:00.222234 (+ 900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:56:00.222239 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:56:00.222249 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:56:00.222257 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:56:00.222306 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:56:00.222320 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:56:00.222442 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 18:56:00.222559 (+ 117us) server_negotiation.cc:299] Negotiation successful
0811 18:56:00.222786 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":115,"threads_started":1}
I20260811 18:56:00.223613 14652 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b8c228182f62487880d38e1203f9bfdd: Neither blocks nor log segments found. Creating new log.
I20260811 18:56:00.224295 14652 log.cc:824] T 00000000000000000000000000000000 P b8c228182f62487880d38e1203f9bfdd: Log is configured to *not* fsync() on all Append() calls
I20260811 18:56:00.226198 14652 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8c228182f62487880d38e1203f9bfdd: No bootstrap required, opened a new log
I20260811 18:56:00.229523 14652 raft_consensus.cc:348] T 00000000000000000000000000000000 P b8c228182f62487880d38e1203f9bfdd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8c228182f62487880d38e1203f9bfdd" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 36091 } }
I20260811 18:56:00.229725 14652 raft_consensus.cc:374] T 00000000000000000000000000000000 P b8c228182f62487880d38e1203f9bfdd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:56:00.229799 14652 raft_consensus.cc:729] T 00000000000000000000000000000000 P b8c228182f62487880d38e1203f9bfdd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8c228182f62487880d38e1203f9bfdd, State: Initialized, Role: FOLLOWER
I20260811 18:56:00.230167 14652 consensus_queue.cc:260] T 00000000000000000000000000000000 P b8c228182f62487880d38e1203f9bfdd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8c228182f62487880d38e1203f9bfdd" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 36091 } }
I20260811 18:56:00.230337 14652 raft_consensus.cc:388] T 00000000000000000000000000000000 P b8c228182f62487880d38e1203f9bfdd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:56:00.230455 14652 raft_consensus.cc:482] T 00000000000000000000000000000000 P b8c228182f62487880d38e1203f9bfdd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:56:00.230567 14652 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b8c228182f62487880d38e1203f9bfdd [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:56:00.231344 14652 raft_consensus.cc:504] T 00000000000000000000000000000000 P b8c228182f62487880d38e1203f9bfdd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8c228182f62487880d38e1203f9bfdd" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 36091 } }
I20260811 18:56:00.231621 14652 leader_election.cc:302] T 00000000000000000000000000000000 P b8c228182f62487880d38e1203f9bfdd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b8c228182f62487880d38e1203f9bfdd; no voters:
I20260811 18:56:00.231848 14652 leader_election.cc:288] T 00000000000000000000000000000000 P b8c228182f62487880d38e1203f9bfdd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:56:00.231958 14657 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b8c228182f62487880d38e1203f9bfdd [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:56:00.232156 14657 raft_consensus.cc:686] T 00000000000000000000000000000000 P b8c228182f62487880d38e1203f9bfdd [term 1 LEADER]: Becoming Leader. State: Replica: b8c228182f62487880d38e1203f9bfdd, State: Running, Role: LEADER
I20260811 18:56:00.232481 14657 consensus_queue.cc:237] T 00000000000000000000000000000000 P b8c228182f62487880d38e1203f9bfdd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8c228182f62487880d38e1203f9bfdd" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 36091 } }
I20260811 18:56:00.232930 14652 sys_catalog.cc:564] T 00000000000000000000000000000000 P b8c228182f62487880d38e1203f9bfdd [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:56:00.233943 14659 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8c228182f62487880d38e1203f9bfdd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b8c228182f62487880d38e1203f9bfdd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8c228182f62487880d38e1203f9bfdd" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 36091 } } }
I20260811 18:56:00.234148 14659 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8c228182f62487880d38e1203f9bfdd [sys.catalog]: This master's current role is: LEADER
I20260811 18:56:00.234270 14662 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8c228182f62487880d38e1203f9bfdd [sys.catalog]: SysCatalogTable state changed. Reason: New leader b8c228182f62487880d38e1203f9bfdd. Latest consensus state: current_term: 1 leader_uuid: "b8c228182f62487880d38e1203f9bfdd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8c228182f62487880d38e1203f9bfdd" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 36091 } } }
I20260811 18:56:00.234375 14662 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8c228182f62487880d38e1203f9bfdd [sys.catalog]: This master's current role is: LEADER
I20260811 18:56:00.234645 14674 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:56:00.237286 14674 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:56:00.242357 14674 catalog_manager.cc:1347] Generated new cluster ID: aa3ac653f3a145879a24937b0abd4f7c
I20260811 18:56:00.242410 14674 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:56:00.252826 14674 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:56:00.253114 14674 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:56:00.267089 14674 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b8c228182f62487880d38e1203f9bfdd: Generated new TSK 0
I20260811 18:56:00.267727 14674 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:56:00.279793 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474496157106-28370-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474496157106-28370-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474496157106-28370-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474496157106-28370-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.180.129:0
--local_ip_for_outbound_sockets=127.27.180.129
--webserver_interface=127.27.180.129
--webserver_port=0
--tserver_master_addrs=127.27.180.190:36091
--builtin_ntp_servers=127.27.180.148:43007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 18:56:00.397324 14693 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:56:00.397553 14693 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:56:00.397606 14693 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:56:00.401470 14693 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:56:00.401515 14693 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:56:00.401585 14693 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.180.129
I20260811 18:56:00.406278 14693 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:43007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474496157106-28370-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474496157106-28370-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.180.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474496157106-28370-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.180.129
--webserver_port=0
--tserver_master_addrs=127.27.180.190:36091
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14693
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.180.129
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474496157106-28370-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:56:00.407210 14693 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:56:00.408439 14693 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:56:00.417361 14700 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:56:00.417368 14704 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:56:00.417392 14699 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:56:00.417667 14693 server_base.cc:1034] running on GCE node
I20260811 18:56:00.418111 14693 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:56:00.418705 14693 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:56:00.419893 14693 hybrid_clock.cc:648] HybridClock initialized: now 1786474560419868 us; error 41 us; skew 500 ppm
I20260811 18:56:00.422260 14693 webserver.cc:460] Webserver started at http://127.27.180.129:36393/ using document root <none> and password file <none>
I20260811 18:56:00.422974 14693 fs_manager.cc:359] Metadata directory not provided
I20260811 18:56:00.423062 14693 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:56:00.423307 14693 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:56:00.426237 14693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474496157106-28370-0/minicluster-data/ts-0/data/instance:
uuid: "ff891a3c47954d9f8d0ccb33f05efe8f"
format_stamp: "Formatted at 2026-08-11 18:56:00 on dist-test-slave-2j7r"
server_key: "af055d134ec5f2cac581524b84670bcd"
server_key_iv: "d47bad51e5137ea69381c51bc2d1bfd8"
server_key_version: "encryptionkey@0"
I20260811 18:56:00.427120 14693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474496157106-28370-0/minicluster-data/ts-0/wal/instance:
uuid: "ff891a3c47954d9f8d0ccb33f05efe8f"
format_stamp: "Formatted at 2026-08-11 18:56:00 on dist-test-slave-2j7r"
server_key: "af055d134ec5f2cac581524b84670bcd"
server_key_iv: "d47bad51e5137ea69381c51bc2d1bfd8"
server_key_version: "encryptionkey@0"
I20260811 18:56:00.431785 14693 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 18:56:00.434646 14711 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:56:00.435845 14693 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 18:56:00.435961 14693 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474496157106-28370-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474496157106-28370-0/minicluster-data/ts-0/wal
uuid: "ff891a3c47954d9f8d0ccb33f05efe8f"
format_stamp: "Formatted at 2026-08-11 18:56:00 on dist-test-slave-2j7r"
server_key: "af055d134ec5f2cac581524b84670bcd"
server_key_iv: "d47bad51e5137ea69381c51bc2d1bfd8"
server_key_version: "encryptionkey@0"
I20260811 18:56:00.436064 14693 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474496157106-28370-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474496157106-28370-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474496157106-28370-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:56:00.440425 14693 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:56:00.441010 14693 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:56:00.441195 14693 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:56:00.441720 14693 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:56:00.442574 14693 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:56:00.442619 14693 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:56:00.442668 14693 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:56:00.442694 14693 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:56:00.450939 14693 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.129:35833
I20260811 18:56:00.450991 14912 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.129:35833 every 8 connection(s)
I20260811 18:56:00.451799 14693 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474496157106-28370-0/minicluster-data/ts-0/data/info.pb
I20260811 18:56:00.457643 14655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:56:00.453539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.129:36037 (local address 127.27.180.190:36091)
0811 18:56:00.453676 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 18:56:00.453680 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:56:00.454778 (+ 1098us) server_negotiation.cc:407] Connection header received
0811 18:56:00.456121 (+ 1343us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:56:00.456125 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:56:00.456168 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:56:00.456260 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:56:00.457288 (+ 1028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:56:00.457291 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:56:00.457293 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:56:00.457305 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:56:00.457324 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:56:00.457331 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:56:00.457413 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 18:56:00.457507 (+ 94us) server_negotiation.cc:299] Negotiation successful
0811 18:56:00.457541 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 18:56:00.458510 14918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:56:00.453900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:36091 (local address 127.27.180.129:36037)
0811 18:56:00.454625 (+ 725us) negotiation.cc:107] Waiting for socket to connect
0811 18:56:00.454668 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 18:56:00.455910 (+ 1242us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:56:00.456402 (+ 492us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:56:00.456408 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:56:00.456494 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:56:00.456971 (+ 477us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:56:00.456973 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:56:00.457013 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:56:00.457080 (+ 67us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:56:00.457402 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:56:00.457408 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:56:00.457412 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 18:56:00.457543 (+ 131us) client_negotiation.cc:240] Negotiation successful
0811 18:56:00.457746 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":106,"threads_started":1}
I20260811 18:56:00.459664 14913 heartbeater.cc:347] Connected to a master server at 127.27.180.190:36091
I20260811 18:56:00.459853 14913 heartbeater.cc:464] Registering TS with master...
I20260811 18:56:00.460201 14913 heartbeater.cc:511] Master 127.27.180.190:36091 requested a full tablet report, sending...
I20260811 18:56:00.461149 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 14693
I20260811 18:56:00.461288 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474496157106-28370-0/minicluster-data/ts-0/wal/instance
I20260811 18:56:00.461619 28370 external_mini_cluster.cc:1428] Setting key 852f773964efd8e0efab7861ae4d21e7
I20260811 18:56:00.461776 14573 ts_manager.cc:194] Registered new tserver with Master: ff891a3c47954d9f8d0ccb33f05efe8f (127.27.180.129:35833)
I20260811 18:56:00.463752 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474496157106-28370-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474496157106-28370-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474496157106-28370-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474496157106-28370-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.180.130:0
--local_ip_for_outbound_sockets=127.27.180.130
--webserver_interface=127.27.180.130
--webserver_port=0
--tserver_master_addrs=127.27.180.190:36091
--builtin_ntp_servers=127.27.180.148:43007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 18:56:00.572166 14919 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:56:00.572393 14919 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:56:00.572440 14919 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:56:00.576330 14919 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:56:00.576383 14919 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:56:00.576443 14919 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.180.130
I20260811 18:56:00.581116 14919 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:43007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474496157106-28370-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474496157106-28370-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.180.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474496157106-28370-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.180.130
--webserver_port=0
--tserver_master_addrs=127.27.180.190:36091
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14919
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.180.130
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474496157106-28370-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:56:00.582163 14919 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:56:00.583508 14919 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:56:00.592821 14931 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:56:00.592859 14927 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:56:00.592922 14934 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:56:00.593061 14919 server_base.cc:1034] running on GCE node
I20260811 18:56:00.593546 14919 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:56:00.594146 14919 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:56:00.595311 14919 hybrid_clock.cc:648] HybridClock initialized: now 1786474560595282 us; error 40 us; skew 500 ppm
I20260811 18:56:00.596912 14919 webserver.cc:460] Webserver started at http://127.27.180.130:39595/ using document root <none> and password file <none>
I20260811 18:56:00.597465 14919 fs_manager.cc:359] Metadata directory not provided
I20260811 18:56:00.597532 14919 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:56:00.597736 14919 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:56:00.599733 14919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474496157106-28370-0/minicluster-data/ts-1/data/instance:
uuid: "59da35ae8dec4275836f84dc0c65a267"
format_stamp: "Formatted at 2026-08-11 18:56:00 on dist-test-slave-2j7r"
server_key: "1e25602cb7666d08480319357eccd44e"
server_key_iv: "e6172b9c197963fe7b1f0d36a7bcdd07"
server_key_version: "encryptionkey@0"
I20260811 18:56:00.600267 14919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474496157106-28370-0/minicluster-data/ts-1/wal/instance:
uuid: "59da35ae8dec4275836f84dc0c65a267"
format_stamp: "Formatted at 2026-08-11 18:56:00 on dist-test-slave-2j7r"
server_key: "1e25602cb7666d08480319357eccd44e"
server_key_iv: "e6172b9c197963fe7b1f0d36a7bcdd07"
server_key_version: "encryptionkey@0"
I20260811 18:56:00.603927 14919 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260811 18:56:00.606344 14946 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:56:00.607484 14919 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 18:56:00.607594 14919 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474496157106-28370-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474496157106-28370-0/minicluster-data/ts-1/wal
uuid: "59da35ae8dec4275836f84dc0c65a267"
format_stamp: "Formatted at 2026-08-11 18:56:00 on dist-test-slave-2j7r"
server_key: "1e25602cb7666d08480319357eccd44e"
server_key_iv: "e6172b9c197963fe7b1f0d36a7bcdd07"
server_key_version: "encryptionkey@0"
I20260811 18:56:00.607689 14919 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474496157106-28370-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474496157106-28370-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474496157106-28370-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:56:00.612146 14919 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:56:00.612735 14919 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:56:00.612884 14919 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:56:00.613442 14919 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:56:00.614437 14919 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:56:00.614485 14919 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:56:00.614534 14919 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:56:00.614562 14919 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:56:00.622624 14919 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.130:44109
I20260811 18:56:00.622658 15162 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.130:44109 every 8 connection(s)
I20260811 18:56:00.623445 14919 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474496157106-28370-0/minicluster-data/ts-1/data/info.pb
I20260811 18:56:00.629240 14655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:56:00.625208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.130:44507 (local address 127.27.180.190:36091)
0811 18:56:00.625331 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0811 18:56:00.625334 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:56:00.626396 (+ 1062us) server_negotiation.cc:407] Connection header received
0811 18:56:00.627667 (+ 1271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:56:00.627670 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:56:00.627711 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:56:00.627798 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:56:00.628731 (+ 933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:56:00.628734 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:56:00.628735 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:56:00.628748 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:56:00.628768 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:56:00.628776 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:56:00.628850 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 18:56:00.628990 (+ 140us) server_negotiation.cc:299] Negotiation successful
0811 18:56:00.629038 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 18:56:00.629976 15166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:56:00.625589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:36091 (local address 127.27.180.130:44507)
0811 18:56:00.626270 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0811 18:56:00.626309 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 18:56:00.627453 (+ 1144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:56:00.627936 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:56:00.627943 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:56:00.628027 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:56:00.628484 (+ 457us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:56:00.628485 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:56:00.628522 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:56:00.628588 (+ 66us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:56:00.628839 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:56:00.628843 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:56:00.628847 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 18:56:00.628967 (+ 120us) client_negotiation.cc:240] Negotiation successful
0811 18:56:00.629189 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":128,"threads_started":1}
I20260811 18:56:00.630143 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 14919
I20260811 18:56:00.630265 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474496157106-28370-0/minicluster-data/ts-1/wal/instance
I20260811 18:56:00.630604 28370 external_mini_cluster.cc:1428] Setting key 340f4a069d4c47226229331f54e6fe64
I20260811 18:56:00.631448 15163 heartbeater.cc:347] Connected to a master server at 127.27.180.190:36091
I20260811 18:56:00.631732 15163 heartbeater.cc:464] Registering TS with master...
I20260811 18:56:00.632179 15163 heartbeater.cc:511] Master 127.27.180.190:36091 requested a full tablet report, sending...
I20260811 18:56:00.632779 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474496157106-28370-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474496157106-28370-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474496157106-28370-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474496157106-28370-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.180.131:0
--local_ip_for_outbound_sockets=127.27.180.131
--webserver_interface=127.27.180.131
--webserver_port=0
--tserver_master_addrs=127.27.180.190:36091
--builtin_ntp_servers=127.27.180.148:43007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260811 18:56:00.634944 14572 ts_manager.cc:194] Registered new tserver with Master: 59da35ae8dec4275836f84dc0c65a267 (127.27.180.130:44109)
W20260811 18:56:00.745975 15167 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:56:00.746248 15167 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:56:00.746304 15167 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:56:00.750368 15167 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:56:00.750419 15167 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:56:00.750494 15167 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.180.131
I20260811 18:56:00.755338 15167 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:43007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474496157106-28370-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474496157106-28370-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.180.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474496157106-28370-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.180.131
--webserver_port=0
--tserver_master_addrs=127.27.180.190:36091
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15167
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.180.131
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474496157106-28370-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:56:00.756363 15167 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:56:00.757735 15167 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:56:00.766822 15175 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:56:00.766849 15177 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:56:00.766830 15174 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:56:00.767227 15167 server_base.cc:1034] running on GCE node
I20260811 18:56:00.767599 15167 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:56:00.768198 15167 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:56:00.769356 15167 hybrid_clock.cc:648] HybridClock initialized: now 1786474560769325 us; error 48 us; skew 500 ppm
I20260811 18:56:00.771099 15167 webserver.cc:460] Webserver started at http://127.27.180.131:46049/ using document root <none> and password file <none>
I20260811 18:56:00.771615 15167 fs_manager.cc:359] Metadata directory not provided
I20260811 18:56:00.771673 15167 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:56:00.771852 15167 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:56:00.773844 15167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474496157106-28370-0/minicluster-data/ts-2/data/instance:
uuid: "6e6d565ece8848f6acc3f53f6eef486e"
format_stamp: "Formatted at 2026-08-11 18:56:00 on dist-test-slave-2j7r"
server_key: "cbd918b39d644f4f2849998bdf0c37de"
server_key_iv: "8c311f5a45ee5f7fd27914ae49ff59ac"
server_key_version: "encryptionkey@0"
I20260811 18:56:00.774340 15167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474496157106-28370-0/minicluster-data/ts-2/wal/instance:
uuid: "6e6d565ece8848f6acc3f53f6eef486e"
format_stamp: "Formatted at 2026-08-11 18:56:00 on dist-test-slave-2j7r"
server_key: "cbd918b39d644f4f2849998bdf0c37de"
server_key_iv: "8c311f5a45ee5f7fd27914ae49ff59ac"
server_key_version: "encryptionkey@0"
I20260811 18:56:00.777879 15167 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 18:56:00.780215 15185 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:56:00.781423 15167 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 18:56:00.781525 15167 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474496157106-28370-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474496157106-28370-0/minicluster-data/ts-2/wal
uuid: "6e6d565ece8848f6acc3f53f6eef486e"
format_stamp: "Formatted at 2026-08-11 18:56:00 on dist-test-slave-2j7r"
server_key: "cbd918b39d644f4f2849998bdf0c37de"
server_key_iv: "8c311f5a45ee5f7fd27914ae49ff59ac"
server_key_version: "encryptionkey@0"
I20260811 18:56:00.781625 15167 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474496157106-28370-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474496157106-28370-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474496157106-28370-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:56:00.785964 15167 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:56:00.786525 15167 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:56:00.786672 15167 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:56:00.787202 15167 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:56:00.788085 15167 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:56:00.788132 15167 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:56:00.788173 15167 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:56:00.788201 15167 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:56:00.796619 15167 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.131:34869
I20260811 18:56:00.796648 15390 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.131:34869 every 8 connection(s)
I20260811 18:56:00.797503 15167 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474496157106-28370-0/minicluster-data/ts-2/data/info.pb
I20260811 18:56:00.800050 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 15167
I20260811 18:56:00.800155 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786474496157106-28370-0/minicluster-data/ts-2/wal/instance
I20260811 18:56:00.800474 28370 external_mini_cluster.cc:1428] Setting key e1f33299b74e65650263b3a1f5261df4
I20260811 18:56:00.803545 14655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:56:00.799168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.131:56675 (local address 127.27.180.190:36091)
0811 18:56:00.799338 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0811 18:56:00.799344 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:56:00.800376 (+ 1032us) server_negotiation.cc:407] Connection header received
0811 18:56:00.801890 (+ 1514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:56:00.801894 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:56:00.801939 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:56:00.802012 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:56:00.803091 (+ 1079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:56:00.803095 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:56:00.803098 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:56:00.803110 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:56:00.803131 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:56:00.803138 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:56:00.803209 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 18:56:00.803340 (+ 131us) server_negotiation.cc:299] Negotiation successful
0811 18:56:00.803412 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 18:56:00.804327 15393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:56:00.799552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:36091 (local address 127.27.180.131:56675)
0811 18:56:00.800244 (+ 692us) negotiation.cc:107] Waiting for socket to connect
0811 18:56:00.800283 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 18:56:00.801683 (+ 1400us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:56:00.802180 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:56:00.802186 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:56:00.802271 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:56:00.802842 (+ 571us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:56:00.802844 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:56:00.802882 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:56:00.802958 (+ 76us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:56:00.803207 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:56:00.803216 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:56:00.803222 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 18:56:00.803381 (+ 159us) client_negotiation.cc:240] Negotiation successful
0811 18:56:00.803610 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":142,"threads_started":1}
I20260811 18:56:00.805326 15391 heartbeater.cc:347] Connected to a master server at 127.27.180.190:36091
I20260811 18:56:00.805505 15391 heartbeater.cc:464] Registering TS with master...
I20260811 18:56:00.805838 15391 heartbeater.cc:511] Master 127.27.180.190:36091 requested a full tablet report, sending...
I20260811 18:56:00.806756 14572 ts_manager.cc:194] Registered new tserver with Master: 6e6d565ece8848f6acc3f53f6eef486e (127.27.180.131:34869)
I20260811 18:56:00.814038 28370 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 18:56:00.816642 14655 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:56:00.815847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56432 (local address 127.27.180.190:36091)
0811 18:56:00.815964 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0811 18:56:00.815968 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:56:00.816121 (+ 153us) server_negotiation.cc:407] Connection header received
0811 18:56:00.816224 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:56:00.816227 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:56:00.816262 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:56:00.816324 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:56:00.816553 (+ 229us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56432: BlockingRecv error: recv got EOF from 127.0.0.1:56432 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 18:56:00.816919 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 14693
I20260811 18:56:00.834477 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 14919
I20260811 18:56:00.862323 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 15167
I20260811 18:56:00.881731 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 14524
2026-08-11T18:56:00Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (868 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474496157106-28370-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:56:00 dist-test-slave-2j7r krb5kdc[15411](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:56:00 dist-test-slave-2j7r krb5kdc[15411](info): set up 2 sockets
Aug 11 18:56:00 dist-test-slave-2j7r krb5kdc[15411](info): commencing operation
krb5kdc: starting...
W20260811 18:56:02.096436 28370 mini_kdc.cc:121] Time spent starting KDC: real 1.184s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474496157106-28370-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474496157106-28370-0/krb5kdc/test-user.keytab.
Aug 11 18:56:02 dist-test-slave-2j7r krb5kdc[15411](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474562, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:56:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:56:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.180.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.180.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.180.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab.
Entry for principal kudu/127.27.180.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab.
I20260811 18:56:02.214239 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474496157106-28370-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474496157106-28370-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474496157106-28370-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.190:35581
--webserver_interface=127.27.180.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab
--principal=kudu/127.27.180.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:46379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.180.190:35581
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:56:02.321827 15431 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:56:02.322075 15431 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:56:02.322131 15431 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:56:02.326081 15431 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:56:02.326130 15431 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:56:02.326155 15431 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:56:02.326177 15431 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:56:02.326206 15431 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:56:02.331290 15431 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:46379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474496157106-28370-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474496157106-28370-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.180.190:35581
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab
--principal=kudu/127.27.180.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.190:35581
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.180.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15431
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474496157106-28370-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:56:02.332261 15431 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:56:02.333274 15431 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:56:02.341323 15438 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:56:02.341329 15448 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:56:02.341329 15440 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:56:02.341631 15431 server_base.cc:1034] running on GCE node
I20260811 18:56:02.342159 15431 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:56:02.343197 15431 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:56:02.344374 15431 hybrid_clock.cc:648] HybridClock initialized: now 1786474562344357 us; error 28 us; skew 500 ppm
Aug 11 18:56:02 dist-test-slave-2j7r krb5kdc[15411](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474562, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:56:02.346977 15431 init.cc:377] Logged in from keytab as kudu/127.27.180.190@KRBTEST.COM (short username kudu)
I20260811 18:56:02.347863 15431 webserver.cc:460] Webserver started at http://127.27.180.190:44765/ using document root <none> and password file <none>
I20260811 18:56:02.348385 15431 fs_manager.cc:359] Metadata directory not provided
I20260811 18:56:02.348449 15431 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:56:02.348628 15431 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:56:02.350613 15431 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474496157106-28370-0/minicluster-data/master-0/data/instance:
uuid: "5390ce0a0c714095b34a0f12eddbd5fb"
format_stamp: "Formatted at 2026-08-11 18:56:02 on dist-test-slave-2j7r"
server_key: "5f121b2479c61c9b8f7b58559186d0be"
server_key_iv: "b84487e06a7d3f6d77c1210c73f37a64"
server_key_version: "encryptionkey@0"
I20260811 18:56:02.351150 15431 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474496157106-28370-0/minicluster-data/master-0/wal/instance:
uuid: "5390ce0a0c714095b34a0f12eddbd5fb"
format_stamp: "Formatted at 2026-08-11 18:56:02 on dist-test-slave-2j7r"
server_key: "5f121b2479c61c9b8f7b58559186d0be"
server_key_iv: "b84487e06a7d3f6d77c1210c73f37a64"
server_key_version: "encryptionkey@0"
I20260811 18:56:02.354686 15431 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 18:56:02.356891 15458 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:56:02.358075 15431 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 18:56:02.358198 15431 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474496157106-28370-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474496157106-28370-0/minicluster-data/master-0/wal
uuid: "5390ce0a0c714095b34a0f12eddbd5fb"
format_stamp: "Formatted at 2026-08-11 18:56:02 on dist-test-slave-2j7r"
server_key: "5f121b2479c61c9b8f7b58559186d0be"
server_key_iv: "b84487e06a7d3f6d77c1210c73f37a64"
server_key_version: "encryptionkey@0"
I20260811 18:56:02.358310 15431 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474496157106-28370-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474496157106-28370-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474496157106-28370-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:56:02.391038 15431 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:56:02.393766 15431 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:56:02.393947 15431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:56:02.400799 15431 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.190:35581
I20260811 18:56:02.401062 15555 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.190:35581 every 8 connection(s)
I20260811 18:56:02.401762 15431 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
I20260811 18:56:02.404474 15556 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:56:02.409839 15556 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5390ce0a0c714095b34a0f12eddbd5fb: Bootstrap starting.
I20260811 18:56:02.410393 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 15431
I20260811 18:56:02.410509 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474496157106-28370-0/minicluster-data/master-0/wal/instance
I20260811 18:56:02.410784 28370 external_mini_cluster.cc:1428] Setting key 7538310e53ec36b1a551727fbbacfa94
I20260811 18:56:02.412459 15556 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5390ce0a0c714095b34a0f12eddbd5fb: Neither blocks nor log segments found. Creating new log.
I20260811 18:56:02.413239 15556 log.cc:824] T 00000000000000000000000000000000 P 5390ce0a0c714095b34a0f12eddbd5fb: Log is configured to *not* fsync() on all Append() calls
I20260811 18:56:02.415019 15556 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5390ce0a0c714095b34a0f12eddbd5fb: No bootstrap required, opened a new log
I20260811 18:56:02.418479 15556 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5390ce0a0c714095b34a0f12eddbd5fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5390ce0a0c714095b34a0f12eddbd5fb" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 35581 } }
I20260811 18:56:02.418691 15556 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5390ce0a0c714095b34a0f12eddbd5fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:56:02.418751 15556 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5390ce0a0c714095b34a0f12eddbd5fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5390ce0a0c714095b34a0f12eddbd5fb, State: Initialized, Role: FOLLOWER
Aug 11 18:56:02 dist-test-slave-2j7r krb5kdc[15411](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474562, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM
I20260811 18:56:02.419188 15556 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5390ce0a0c714095b34a0f12eddbd5fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5390ce0a0c714095b34a0f12eddbd5fb" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 35581 } }
I20260811 18:56:02.419355 15556 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5390ce0a0c714095b34a0f12eddbd5fb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:56:02.419426 15556 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5390ce0a0c714095b34a0f12eddbd5fb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:56:02.419529 15556 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5390ce0a0c714095b34a0f12eddbd5fb [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:56:02.420480 15556 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5390ce0a0c714095b34a0f12eddbd5fb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5390ce0a0c714095b34a0f12eddbd5fb" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 35581 } }
I20260811 18:56:02.420823 15556 leader_election.cc:302] T 00000000000000000000000000000000 P 5390ce0a0c714095b34a0f12eddbd5fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5390ce0a0c714095b34a0f12eddbd5fb; no voters:
I20260811 18:56:02.421162 15556 leader_election.cc:288] T 00000000000000000000000000000000 P 5390ce0a0c714095b34a0f12eddbd5fb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:56:02.421241 15564 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5390ce0a0c714095b34a0f12eddbd5fb [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:56:02.421455 15564 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5390ce0a0c714095b34a0f12eddbd5fb [term 1 LEADER]: Becoming Leader. State: Replica: 5390ce0a0c714095b34a0f12eddbd5fb, State: Running, Role: LEADER
I20260811 18:56:02.421808 15564 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5390ce0a0c714095b34a0f12eddbd5fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5390ce0a0c714095b34a0f12eddbd5fb" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 35581 } }
I20260811 18:56:02.422421 15556 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5390ce0a0c714095b34a0f12eddbd5fb [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:56:02.422945 15564 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5390ce0a0c714095b34a0f12eddbd5fb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5390ce0a0c714095b34a0f12eddbd5fb. Latest consensus state: current_term: 1 leader_uuid: "5390ce0a0c714095b34a0f12eddbd5fb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5390ce0a0c714095b34a0f12eddbd5fb" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 35581 } } }
I20260811 18:56:02.423056 15564 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5390ce0a0c714095b34a0f12eddbd5fb [sys.catalog]: This master's current role is: LEADER
I20260811 18:56:02.423728 15565 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5390ce0a0c714095b34a0f12eddbd5fb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5390ce0a0c714095b34a0f12eddbd5fb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5390ce0a0c714095b34a0f12eddbd5fb" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 35581 } } }
I20260811 18:56:02.423828 15565 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5390ce0a0c714095b34a0f12eddbd5fb [sys.catalog]: This master's current role is: LEADER
I20260811 18:56:02.424867 15576 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:56:02.425668 15559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:56:02.411954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45246 (local address 127.27.180.190:35581)
0811 18:56:02.412341 (+ 387us) server_negotiation.cc:206] Beginning negotiation
0811 18:56:02.412357 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 18:56:02.412392 (+ 35us) server_negotiation.cc:407] Connection header received
0811 18:56:02.413066 (+ 674us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:56:02.413110 (+ 44us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:56:02.413465 (+ 355us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:56:02.413767 (+ 302us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:56:02.414571 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:56:02.415578 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:56:02.416399 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:56:02.416865 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:56:02.417010 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:56:02.419748 (+ 2738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:56:02.419753 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:56:02.419767 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:56:02.419796 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:56:02.421683 (+ 1887us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:56:02.422271 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:56:02.422278 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:56:02.422285 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:56:02.422413 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:56:02.422716 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:56:02.422720 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:56:02.422722 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:56:02.423347 (+ 625us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:56:02.424219 (+ 872us) server_negotiation.cc:1035] Waiting for connection context
0811 18:56:02.424410 (+ 191us) server_negotiation.cc:299] Negotiation successful
0811 18:56:02.424712 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":257,"thread_start_us":105,"threads_started":1}
I20260811 18:56:02.427925 15576 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:56:02.433449 15576 catalog_manager.cc:1347] Generated new cluster ID: 211769f742ab4f1c89926af6ce4013d1
I20260811 18:56:02.433503 15576 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:56:02.452162 15576 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:56:02.453075 15576 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:56:02.463326 15576 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5390ce0a0c714095b34a0f12eddbd5fb: Generated new TSK 0
I20260811 18:56:02.463966 15576 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.180.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.180.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.180.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.129.keytab.
Entry for principal kudu/127.27.180.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.129.keytab.
I20260811 18:56:02.502125 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474496157106-28370-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474496157106-28370-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474496157106-28370-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474496157106-28370-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.180.129:0
--local_ip_for_outbound_sockets=127.27.180.129
--webserver_interface=127.27.180.129
--webserver_port=0
--tserver_master_addrs=127.27.180.190:35581
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.129.keytab
--principal=kudu/127.27.180.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:46379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:56:02.612613 15602 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:56:02.612836 15602 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:56:02.612875 15602 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:56:02.616782 15602 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:56:02.616833 15602 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:56:02.616892 15602 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.180.129
I20260811 18:56:02.621941 15602 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:46379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474496157106-28370-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474496157106-28370-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.129.keytab
--principal=kudu/127.27.180.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.180.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474496157106-28370-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.180.129
--webserver_port=0
--tserver_master_addrs=127.27.180.190:35581
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15602
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.180.129
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474496157106-28370-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:56:02.622900 15602 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:56:02.623927 15602 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:56:02.633313 15622 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:56:02.633349 15612 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:56:02.633320 15610 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:56:02.633498 15602 server_base.cc:1034] running on GCE node
I20260811 18:56:02.633963 15602 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:56:02.634567 15602 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:56:02.635727 15602 hybrid_clock.cc:648] HybridClock initialized: now 1786474562635718 us; error 32 us; skew 500 ppm
Aug 11 18:56:02 dist-test-slave-2j7r krb5kdc[15411](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474562, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:56:02.638804 15602 init.cc:377] Logged in from keytab as kudu/127.27.180.129@KRBTEST.COM (short username kudu)
I20260811 18:56:02.639791 15602 webserver.cc:460] Webserver started at http://127.27.180.129:42661/ using document root <none> and password file <none>
I20260811 18:56:02.640471 15602 fs_manager.cc:359] Metadata directory not provided
I20260811 18:56:02.640551 15602 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:56:02.640769 15602 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:56:02.642843 15602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474496157106-28370-0/minicluster-data/ts-0/data/instance:
uuid: "d13a1a0caf8c4cd88caa7c1a3d210ee3"
format_stamp: "Formatted at 2026-08-11 18:56:02 on dist-test-slave-2j7r"
server_key: "7218609c95a54158fd6ec74fd947ea6b"
server_key_iv: "3641f0dd461a687f9b36df739bd46be9"
server_key_version: "encryptionkey@0"
I20260811 18:56:02.643421 15602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474496157106-28370-0/minicluster-data/ts-0/wal/instance:
uuid: "d13a1a0caf8c4cd88caa7c1a3d210ee3"
format_stamp: "Formatted at 2026-08-11 18:56:02 on dist-test-slave-2j7r"
server_key: "7218609c95a54158fd6ec74fd947ea6b"
server_key_iv: "3641f0dd461a687f9b36df739bd46be9"
server_key_version: "encryptionkey@0"
I20260811 18:56:02.647230 15602 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 18:56:02.649724 15638 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:56:02.650851 15602 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 18:56:02.650967 15602 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474496157106-28370-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474496157106-28370-0/minicluster-data/ts-0/wal
uuid: "d13a1a0caf8c4cd88caa7c1a3d210ee3"
format_stamp: "Formatted at 2026-08-11 18:56:02 on dist-test-slave-2j7r"
server_key: "7218609c95a54158fd6ec74fd947ea6b"
server_key_iv: "3641f0dd461a687f9b36df739bd46be9"
server_key_version: "encryptionkey@0"
I20260811 18:56:02.651067 15602 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474496157106-28370-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474496157106-28370-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474496157106-28370-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:56:02.666980 15602 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:56:02.669957 15602 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:56:02.670140 15602 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:56:02.670719 15602 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:56:02.671562 15602 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:56:02.671615 15602 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:56:02.671682 15602 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:56:02.671710 15602 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:56:02.680860 15602 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.129:43171
I20260811 18:56:02.680881 15839 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.129:43171 every 8 connection(s)
I20260811 18:56:02.681814 15602 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474496157106-28370-0/minicluster-data/ts-0/data/info.pb
I20260811 18:56:02.688357 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 15602
I20260811 18:56:02.688483 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474496157106-28370-0/minicluster-data/ts-0/wal/instance
I20260811 18:56:02.688741 28370 external_mini_cluster.cc:1428] Setting key 58324ab6bf8f6b72d744ed65f36dc041
Aug 11 18:56:02 dist-test-slave-2j7r krb5kdc[15411](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474562, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.129@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM
I20260811 18:56:02.696080 15559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:56:02.683567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.129:53841 (local address 127.27.180.190:35581)
0811 18:56:02.683715 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 18:56:02.683719 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:56:02.684504 (+ 785us) server_negotiation.cc:407] Connection header received
0811 18:56:02.685960 (+ 1456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:56:02.685965 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:56:02.686030 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:56:02.686107 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:56:02.687376 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:56:02.687917 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:56:02.689187 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:56:02.689459 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:56:02.689546 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:56:02.692302 (+ 2756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:56:02.692306 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:56:02.692309 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:56:02.692338 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:56:02.693907 (+ 1569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:56:02.694453 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:56:02.694456 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:56:02.694458 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:56:02.694548 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:56:02.694886 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:56:02.694889 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:56:02.694890 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:56:02.695115 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:56:02.695233 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 18:56:02.695783 (+ 550us) server_negotiation.cc:299] Negotiation successful
0811 18:56:02.695916 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 18:56:02.697029 15842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:56:02.683881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:35581 (local address 127.27.180.129:53841)
0811 18:56:02.684344 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0811 18:56:02.684387 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 18:56:02.685727 (+ 1340us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:56:02.686309 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:56:02.686316 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:56:02.686683 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:56:02.687208 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:56:02.687224 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:56:02.688055 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:56:02.688058 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:56:02.689024 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:56:02.689035 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:56:02.689615 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:56:02.689620 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:56:02.689836 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:56:02.690431 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:56:02.690448 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:56:02.692107 (+ 1659us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:56:02.694025 (+ 1918us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:56:02.694030 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:56:02.694041 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:56:02.694345 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:56:02.694645 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:56:02.694648 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:56:02.694649 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:56:02.694798 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:56:02.695249 (+ 451us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:56:02.695254 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:56:02.695529 (+ 275us) client_negotiation.cc:769] Sending connection context
0811 18:56:02.695710 (+ 181us) client_negotiation.cc:240] Negotiation successful
0811 18:56:02.695965 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":126,"threads_started":1}
I20260811 18:56:02.698366 15840 heartbeater.cc:347] Connected to a master server at 127.27.180.190:35581
I20260811 18:56:02.698632 15840 heartbeater.cc:464] Registering TS with master...
I20260811 18:56:02.699250 15840 heartbeater.cc:511] Master 127.27.180.190:35581 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.27.180.130@KRBTEST.COM; defaulting to no policy
I20260811 18:56:02.701002 15497 ts_manager.cc:194] Registered new tserver with Master: d13a1a0caf8c4cd88caa7c1a3d210ee3 (127.27.180.129:43171)
I20260811 18:56:02.702313 15497 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.180.129@KRBTEST.COM'} at 127.27.180.129:53841
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.180.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.180.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.130.keytab.
Entry for principal kudu/127.27.180.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.130.keytab.
I20260811 18:56:02.719561 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474496157106-28370-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474496157106-28370-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474496157106-28370-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474496157106-28370-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.180.130:0
--local_ip_for_outbound_sockets=127.27.180.130
--webserver_interface=127.27.180.130
--webserver_port=0
--tserver_master_addrs=127.27.180.190:35581
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.130.keytab
--principal=kudu/127.27.180.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:46379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:56:02.828744 15846 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:56:02.828965 15846 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:56:02.829020 15846 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:56:02.833138 15846 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:56:02.833190 15846 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:56:02.833273 15846 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.180.130
I20260811 18:56:02.838239 15846 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:46379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474496157106-28370-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474496157106-28370-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.130.keytab
--principal=kudu/127.27.180.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.180.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474496157106-28370-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.180.130
--webserver_port=0
--tserver_master_addrs=127.27.180.190:35581
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15846
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.180.130
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474496157106-28370-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:56:02.839128 15846 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:56:02.840040 15846 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:56:02.849310 15855 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:56:02.849295 15863 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:56:02.849283 15858 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:56:02.849458 15846 server_base.cc:1034] running on GCE node
I20260811 18:56:02.849920 15846 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:56:02.850499 15846 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:56:02.851651 15846 hybrid_clock.cc:648] HybridClock initialized: now 1786474562851641 us; error 28 us; skew 500 ppm
Aug 11 18:56:02 dist-test-slave-2j7r krb5kdc[15411](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474562, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:56:02.854212 15846 init.cc:377] Logged in from keytab as kudu/127.27.180.130@KRBTEST.COM (short username kudu)
I20260811 18:56:02.855073 15846 webserver.cc:460] Webserver started at http://127.27.180.130:44499/ using document root <none> and password file <none>
I20260811 18:56:02.855561 15846 fs_manager.cc:359] Metadata directory not provided
I20260811 18:56:02.855614 15846 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:56:02.855778 15846 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:56:02.857720 15846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474496157106-28370-0/minicluster-data/ts-1/data/instance:
uuid: "12ab58de0d504260b8ea8ee564d02369"
format_stamp: "Formatted at 2026-08-11 18:56:02 on dist-test-slave-2j7r"
server_key: "31b32417fa9ed1b59a9a9ddcabb05e49"
server_key_iv: "7356b8404de733f0617ee235f4d34656"
server_key_version: "encryptionkey@0"
I20260811 18:56:02.858203 15846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474496157106-28370-0/minicluster-data/ts-1/wal/instance:
uuid: "12ab58de0d504260b8ea8ee564d02369"
format_stamp: "Formatted at 2026-08-11 18:56:02 on dist-test-slave-2j7r"
server_key: "31b32417fa9ed1b59a9a9ddcabb05e49"
server_key_iv: "7356b8404de733f0617ee235f4d34656"
server_key_version: "encryptionkey@0"
I20260811 18:56:02.861655 15846 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:56:02.864573 15874 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:56:02.866092 15846 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:56:02.866202 15846 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474496157106-28370-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474496157106-28370-0/minicluster-data/ts-1/wal
uuid: "12ab58de0d504260b8ea8ee564d02369"
format_stamp: "Formatted at 2026-08-11 18:56:02 on dist-test-slave-2j7r"
server_key: "31b32417fa9ed1b59a9a9ddcabb05e49"
server_key_iv: "7356b8404de733f0617ee235f4d34656"
server_key_version: "encryptionkey@0"
I20260811 18:56:02.866308 15846 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474496157106-28370-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474496157106-28370-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474496157106-28370-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:56:02.884994 15846 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:56:02.887892 15846 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:56:02.888054 15846 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:56:02.888581 15846 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:56:02.889387 15846 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:56:02.889433 15846 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:56:02.889477 15846 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:56:02.889504 15846 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:56:02.897703 15846 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.130:36347
I20260811 18:56:02.897728 16077 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.130:36347 every 8 connection(s)
I20260811 18:56:02.898511 15846 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474496157106-28370-0/minicluster-data/ts-1/data/info.pb
I20260811 18:56:02.905689 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 15846
I20260811 18:56:02.905797 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474496157106-28370-0/minicluster-data/ts-1/wal/instance
I20260811 18:56:02.906188 28370 external_mini_cluster.cc:1428] Setting key 1b990e3dd0b4fb9fb0b0b7f6819a7463
Aug 11 18:56:02 dist-test-slave-2j7r krb5kdc[15411](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474562, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.130@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM
I20260811 18:56:02.911489 15559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:56:02.900202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.130:44819 (local address 127.27.180.190:35581)
0811 18:56:02.900336 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0811 18:56:02.900340 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:56:02.901110 (+ 770us) server_negotiation.cc:407] Connection header received
0811 18:56:02.902454 (+ 1344us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:56:02.902457 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:56:02.902513 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:56:02.902587 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:56:02.903664 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:56:02.904188 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:56:02.905308 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:56:02.905522 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:56:02.905610 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:56:02.908093 (+ 2483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:56:02.908096 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:56:02.908098 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:56:02.908124 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:56:02.909431 (+ 1307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:56:02.909994 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:56:02.909999 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:56:02.910000 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:56:02.910105 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:56:02.910411 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:56:02.910414 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:56:02.910416 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:56:02.910645 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:56:02.910772 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0811 18:56:02.911190 (+ 418us) server_negotiation.cc:299] Negotiation successful
0811 18:56:02.911313 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 18:56:02.912138 16082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:56:02.900537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:35581 (local address 127.27.180.130:44819)
0811 18:56:02.900953 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0811 18:56:02.900993 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 18:56:02.902261 (+ 1268us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:56:02.902757 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:56:02.902764 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:56:02.903085 (+ 321us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:56:02.903527 (+ 442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:56:02.903540 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:56:02.904305 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:56:02.904308 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:56:02.905192 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:56:02.905202 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:56:02.905655 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:56:02.905659 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:56:02.905824 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:56:02.906348 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:56:02.906364 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:56:02.907918 (+ 1554us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:56:02.909564 (+ 1646us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:56:02.909571 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:56:02.909585 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:56:02.909900 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:56:02.910194 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:56:02.910196 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:56:02.910198 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:56:02.910331 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:56:02.910749 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:56:02.910753 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:56:02.910960 (+ 207us) client_negotiation.cc:769] Sending connection context
0811 18:56:02.911100 (+ 140us) client_negotiation.cc:240] Negotiation successful
0811 18:56:02.911345 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":115,"threads_started":1}
I20260811 18:56:02.913200 16078 heartbeater.cc:347] Connected to a master server at 127.27.180.190:35581
I20260811 18:56:02.913762 16078 heartbeater.cc:464] Registering TS with master...
I20260811 18:56:02.914511 16078 heartbeater.cc:511] Master 127.27.180.190:35581 requested a full tablet report, sending...
I20260811 18:56:02.915634 15497 ts_manager.cc:194] Registered new tserver with Master: 12ab58de0d504260b8ea8ee564d02369 (127.27.180.130:36347)
I20260811 18:56:02.916198 15497 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.180.130@KRBTEST.COM'} at 127.27.180.130:44819
WARNING: no policy specified for kudu/127.27.180.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.180.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.180.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.131.keytab.
Entry for principal kudu/127.27.180.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.131.keytab.
I20260811 18:56:02.941174 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474496157106-28370-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474496157106-28370-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474496157106-28370-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474496157106-28370-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.180.131:0
--local_ip_for_outbound_sockets=127.27.180.131
--webserver_interface=127.27.180.131
--webserver_port=0
--tserver_master_addrs=127.27.180.190:35581
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.131.keytab
--principal=kudu/127.27.180.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:46379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:56:03.064047 16090 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:56:03.064307 16090 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:56:03.064352 16090 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:56:03.068290 16090 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:56:03.068349 16090 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:56:03.068408 16090 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.180.131
I20260811 18:56:03.073513 16090 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:46379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474496157106-28370-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474496157106-28370-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.131.keytab
--principal=kudu/127.27.180.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.180.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474496157106-28370-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.180.131
--webserver_port=0
--tserver_master_addrs=127.27.180.190:35581
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16090
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.180.131
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474496157106-28370-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:56:03.074455 16090 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:56:03.075390 16090 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:56:03.084616 16099 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:56:03.084704 16090 server_base.cc:1034] running on GCE node
W20260811 18:56:03.084630 16102 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:56:03.084698 16097 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:56:03.085243 16090 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:56:03.085814 16090 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:56:03.086961 16090 hybrid_clock.cc:648] HybridClock initialized: now 1786474563086937 us; error 35 us; skew 500 ppm
Aug 11 18:56:03 dist-test-slave-2j7r krb5kdc[15411](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474563, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:56:03.089525 16090 init.cc:377] Logged in from keytab as kudu/127.27.180.131@KRBTEST.COM (short username kudu)
I20260811 18:56:03.090305 16090 webserver.cc:460] Webserver started at http://127.27.180.131:39379/ using document root <none> and password file <none>
I20260811 18:56:03.090802 16090 fs_manager.cc:359] Metadata directory not provided
I20260811 18:56:03.090867 16090 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:56:03.091049 16090 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:56:03.092971 16090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474496157106-28370-0/minicluster-data/ts-2/data/instance:
uuid: "6eb10e5e19224473a7aa433f1a863fc5"
format_stamp: "Formatted at 2026-08-11 18:56:03 on dist-test-slave-2j7r"
server_key: "0a971aa309e5b623ba194db8a1814610"
server_key_iv: "585300f343e9da0f91abfd29afe9cd08"
server_key_version: "encryptionkey@0"
I20260811 18:56:03.093545 16090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474496157106-28370-0/minicluster-data/ts-2/wal/instance:
uuid: "6eb10e5e19224473a7aa433f1a863fc5"
format_stamp: "Formatted at 2026-08-11 18:56:03 on dist-test-slave-2j7r"
server_key: "0a971aa309e5b623ba194db8a1814610"
server_key_iv: "585300f343e9da0f91abfd29afe9cd08"
server_key_version: "encryptionkey@0"
I20260811 18:56:03.096992 16090 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:56:03.099291 16112 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:56:03.100461 16090 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 18:56:03.100566 16090 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474496157106-28370-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474496157106-28370-0/minicluster-data/ts-2/wal
uuid: "6eb10e5e19224473a7aa433f1a863fc5"
format_stamp: "Formatted at 2026-08-11 18:56:03 on dist-test-slave-2j7r"
server_key: "0a971aa309e5b623ba194db8a1814610"
server_key_iv: "585300f343e9da0f91abfd29afe9cd08"
server_key_version: "encryptionkey@0"
I20260811 18:56:03.100661 16090 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474496157106-28370-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474496157106-28370-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474496157106-28370-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:56:03.130234 16090 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:56:03.132931 16090 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:56:03.133132 16090 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:56:03.133735 16090 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:56:03.134583 16090 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:56:03.134632 16090 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:56:03.134686 16090 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:56:03.134716 16090 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:56:03.143072 16090 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.131:39023
I20260811 18:56:03.143100 16328 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.131:39023 every 8 connection(s)
I20260811 18:56:03.143923 16090 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474496157106-28370-0/minicluster-data/ts-2/data/info.pb
I20260811 18:56:03.147997 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 16090
I20260811 18:56:03.148114 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474496157106-28370-0/minicluster-data/ts-2/wal/instance
I20260811 18:56:03.148404 28370 external_mini_cluster.cc:1428] Setting key 20bd308923cf9c09903367928bab6c3a
Aug 11 18:56:03 dist-test-slave-2j7r krb5kdc[15411](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474563, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.131@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM
I20260811 18:56:03.156782 15559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:56:03.145650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.131:39049 (local address 127.27.180.190:35581)
0811 18:56:03.145799 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0811 18:56:03.145804 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:56:03.146652 (+ 848us) server_negotiation.cc:407] Connection header received
0811 18:56:03.147973 (+ 1321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:56:03.147976 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:56:03.148036 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:56:03.148111 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:56:03.149479 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:56:03.150084 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:56:03.151295 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:56:03.151520 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:56:03.151596 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:56:03.153821 (+ 2225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:56:03.153824 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:56:03.153826 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:56:03.153851 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:56:03.155086 (+ 1235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:56:03.155586 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:56:03.155589 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:56:03.155590 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:56:03.155666 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:56:03.155938 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:56:03.155941 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:56:03.155942 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:56:03.156101 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:56:03.156170 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 18:56:03.156569 (+ 399us) server_negotiation.cc:299] Negotiation successful
0811 18:56:03.156663 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 18:56:03.157650 16333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:56:03.146046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:35581 (local address 127.27.180.131:39049)
0811 18:56:03.146507 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0811 18:56:03.146550 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 18:56:03.147763 (+ 1213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:56:03.148286 (+ 523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:56:03.148294 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:56:03.148685 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:56:03.149331 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:56:03.149353 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:56:03.150213 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:56:03.150217 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:56:03.151174 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:56:03.151185 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:56:03.151630 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:56:03.151633 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:56:03.151764 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:56:03.152230 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:56:03.152247 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:56:03.153661 (+ 1414us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:56:03.155185 (+ 1524us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:56:03.155189 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:56:03.155200 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:56:03.155506 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:56:03.155734 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:56:03.155736 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:56:03.155738 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:56:03.155873 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:56:03.156169 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:56:03.156174 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:56:03.156392 (+ 218us) client_negotiation.cc:769] Sending connection context
0811 18:56:03.156540 (+ 148us) client_negotiation.cc:240] Negotiation successful
0811 18:56:03.156775 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":109,"threads_started":1}
I20260811 18:56:03.158725 16330 heartbeater.cc:347] Connected to a master server at 127.27.180.190:35581
I20260811 18:56:03.158946 16330 heartbeater.cc:464] Registering TS with master...
I20260811 18:56:03.159409 16330 heartbeater.cc:511] Master 127.27.180.190:35581 requested a full tablet report, sending...
I20260811 18:56:03.160385 15497 ts_manager.cc:194] Registered new tserver with Master: 6eb10e5e19224473a7aa433f1a863fc5 (127.27.180.131:39023)
I20260811 18:56:03.160946 15497 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.180.131@KRBTEST.COM'} at 127.27.180.131:39049
I20260811 18:56:03.162379 28370 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 18:56:03.170900 15559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:56:03.164415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45250 (local address 127.27.180.190:35581)
0811 18:56:03.164547 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 18:56:03.164551 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:56:03.164691 (+ 140us) server_negotiation.cc:407] Connection header received
0811 18:56:03.164799 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:56:03.164801 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:56:03.164838 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:56:03.164898 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:56:03.165541 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:56:03.166043 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:56:03.166812 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:56:03.167018 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:56:03.167082 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:56:03.168248 (+ 1166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:56:03.168251 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:56:03.168253 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:56:03.168268 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:56:03.169414 (+ 1146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:56:03.169802 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:56:03.169805 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:56:03.169806 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:56:03.169873 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:56:03.170079 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:56:03.170081 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:56:03.170082 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:56:03.170309 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:56:03.170410 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 18:56:03.170661 (+ 251us) server_negotiation.cc:299] Negotiation successful
0811 18:56:03.170757 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 18:56:03.173911 15496 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45250:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:56:03.176862 15496 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:56:03.190712 16352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:56:03.185767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.129:43171 (local address 127.0.0.1:44608)
0811 18:56:03.186084 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0811 18:56:03.186095 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 18:56:03.186171 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:56:03.186795 (+ 624us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:56:03.186798 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:56:03.186835 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:56:03.187011 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:56:03.187017 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:56:03.188014 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:56:03.188017 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:56:03.189191 (+ 1174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:56:03.189202 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:56:03.190178 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:56:03.190181 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:56:03.190295 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:56:03.190333 (+ 38us) client_negotiation.cc:769] Sending connection context
0811 18:56:03.190435 (+ 102us) client_negotiation.cc:240] Negotiation successful
0811 18:56:03.190552 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"mutex_wait_us":80,"spinlock_wait_cycles":10752,"thread_start_us":92,"threads_started":1}
I20260811 18:56:03.190712 16354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:56:03.185968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44608 (local address 127.27.180.129:43171)
0811 18:56:03.186401 (+ 433us) server_negotiation.cc:206] Beginning negotiation
0811 18:56:03.186415 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 18:56:03.186434 (+ 19us) server_negotiation.cc:407] Connection header received
0811 18:56:03.186496 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:56:03.186500 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:56:03.186644 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:56:03.186917 (+ 273us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:56:03.187143 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:56:03.187893 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:56:03.189330 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:56:03.190063 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:56:03.190178 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:56:03.190318 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0811 18:56:03.190475 (+ 157us) server_negotiation.cc:299] Negotiation successful
0811 18:56:03.190569 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":92,"threads_started":1}
I20260811 18:56:03.191061 16347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:56:03.184870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.131:39023 (local address 127.0.0.1:56170)
0811 18:56:03.185668 (+ 798us) negotiation.cc:107] Waiting for socket to connect
0811 18:56:03.185719 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 18:56:03.185839 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:56:03.186795 (+ 956us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:56:03.186798 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:56:03.186837 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:56:03.186973 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:56:03.186980 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:56:03.188498 (+ 1518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:56:03.188501 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:56:03.189670 (+ 1169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:56:03.189680 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:56:03.190667 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:56:03.190671 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:56:03.190770 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:56:03.190794 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 18:56:03.190841 (+ 47us) client_negotiation.cc:240] Negotiation successful
0811 18:56:03.190919 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":633,"mutex_wait_us":7,"spinlock_wait_cycles":23040,"thread_start_us":84,"threads_started":1}
I20260811 18:56:03.191269 16351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:56:03.185220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43492 (local address 127.27.180.130:36347)
0811 18:56:03.186140 (+ 920us) server_negotiation.cc:206] Beginning negotiation
0811 18:56:03.186151 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 18:56:03.186175 (+ 24us) server_negotiation.cc:407] Connection header received
0811 18:56:03.186313 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:56:03.186319 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:56:03.186604 (+ 285us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:56:03.186758 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:56:03.187109 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:56:03.188356 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:56:03.189689 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:56:03.190499 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:56:03.190626 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:56:03.190761 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 18:56:03.191023 (+ 262us) server_negotiation.cc:299] Negotiation successful
0811 18:56:03.191120 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":812,"thread_start_us":149,"threads_started":1}
I20260811 18:56:03.191737 16350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:56:03.185023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56170 (local address 127.27.180.131:39023)
0811 18:56:03.186337 (+ 1314us) server_negotiation.cc:206] Beginning negotiation
0811 18:56:03.186351 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 18:56:03.186373 (+ 22us) server_negotiation.cc:407] Connection header received
0811 18:56:03.186455 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:56:03.186461 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:56:03.186644 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:56:03.187121 (+ 477us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:56:03.187263 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:56:03.188356 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:56:03.189787 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:56:03.190545 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:56:03.191340 (+ 795us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:56:03.191461 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 18:56:03.191507 (+ 46us) server_negotiation.cc:299] Negotiation successful
0811 18:56:03.191594 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1202,"thread_start_us":186,"threads_started":1}
I20260811 18:56:03.192242 16349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:56:03.185028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.130:36347 (local address 127.0.0.1:43492)
0811 18:56:03.185698 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0811 18:56:03.185744 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 18:56:03.185842 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:56:03.186769 (+ 927us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:56:03.186774 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:56:03.186795 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:56:03.186970 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:56:03.186978 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:56:03.188526 (+ 1548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:56:03.188529 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:56:03.189570 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:56:03.189580 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:56:03.190818 (+ 1238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:56:03.190821 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:56:03.190896 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:56:03.190916 (+ 20us) client_negotiation.cc:769] Sending connection context
0811 18:56:03.190985 (+ 69us) client_negotiation.cc:240] Negotiation successful
0811 18:56:03.191043 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":603,"mutex_wait_us":65,"spinlock_wait_cycles":11904,"thread_start_us":119,"threads_started":1}
I20260811 18:56:03.194109 15720 tablet_service.cc:1467] Processing CreateTablet for tablet 6dce595b948945bfb2a381bd677e070d (DEFAULT_TABLE table=test-table [id=3f618d12b2024987a05406136bc21153]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:56:03.194347 16203 tablet_service.cc:1467] Processing CreateTablet for tablet 6dce595b948945bfb2a381bd677e070d (DEFAULT_TABLE table=test-table [id=3f618d12b2024987a05406136bc21153]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:56:03.195093 15720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6dce595b948945bfb2a381bd677e070d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:56:03.195410 16203 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6dce595b948945bfb2a381bd677e070d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:56:03.195163 15957 tablet_service.cc:1467] Processing CreateTablet for tablet 6dce595b948945bfb2a381bd677e070d (DEFAULT_TABLE table=test-table [id=3f618d12b2024987a05406136bc21153]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:56:03.196112 15957 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6dce595b948945bfb2a381bd677e070d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:56:03.201236 16355 tablet_bootstrap.cc:492] T 6dce595b948945bfb2a381bd677e070d P d13a1a0caf8c4cd88caa7c1a3d210ee3: Bootstrap starting.
I20260811 18:56:03.203358 16355 tablet_bootstrap.cc:654] T 6dce595b948945bfb2a381bd677e070d P d13a1a0caf8c4cd88caa7c1a3d210ee3: Neither blocks nor log segments found. Creating new log.
I20260811 18:56:03.203472 16357 tablet_bootstrap.cc:492] T 6dce595b948945bfb2a381bd677e070d P 12ab58de0d504260b8ea8ee564d02369: Bootstrap starting.
I20260811 18:56:03.204228 16355 log.cc:824] T 6dce595b948945bfb2a381bd677e070d P d13a1a0caf8c4cd88caa7c1a3d210ee3: Log is configured to *not* fsync() on all Append() calls
I20260811 18:56:03.204749 16356 tablet_bootstrap.cc:492] T 6dce595b948945bfb2a381bd677e070d P 6eb10e5e19224473a7aa433f1a863fc5: Bootstrap starting.
I20260811 18:56:03.206143 16355 tablet_bootstrap.cc:492] T 6dce595b948945bfb2a381bd677e070d P d13a1a0caf8c4cd88caa7c1a3d210ee3: No bootstrap required, opened a new log
I20260811 18:56:03.206238 16357 tablet_bootstrap.cc:654] T 6dce595b948945bfb2a381bd677e070d P 12ab58de0d504260b8ea8ee564d02369: Neither blocks nor log segments found. Creating new log.
I20260811 18:56:03.206337 16355 ts_tablet_manager.cc:1397] T 6dce595b948945bfb2a381bd677e070d P d13a1a0caf8c4cd88caa7c1a3d210ee3: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 18:56:03.206955 16356 tablet_bootstrap.cc:654] T 6dce595b948945bfb2a381bd677e070d P 6eb10e5e19224473a7aa433f1a863fc5: Neither blocks nor log segments found. Creating new log.
I20260811 18:56:03.207118 16357 log.cc:824] T 6dce595b948945bfb2a381bd677e070d P 12ab58de0d504260b8ea8ee564d02369: Log is configured to *not* fsync() on all Append() calls
I20260811 18:56:03.208076 16356 log.cc:824] T 6dce595b948945bfb2a381bd677e070d P 6eb10e5e19224473a7aa433f1a863fc5: Log is configured to *not* fsync() on all Append() calls
I20260811 18:56:03.209204 16357 tablet_bootstrap.cc:492] T 6dce595b948945bfb2a381bd677e070d P 12ab58de0d504260b8ea8ee564d02369: No bootstrap required, opened a new log
I20260811 18:56:03.209391 16357 ts_tablet_manager.cc:1397] T 6dce595b948945bfb2a381bd677e070d P 12ab58de0d504260b8ea8ee564d02369: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 18:56:03.210423 16356 tablet_bootstrap.cc:492] T 6dce595b948945bfb2a381bd677e070d P 6eb10e5e19224473a7aa433f1a863fc5: No bootstrap required, opened a new log
I20260811 18:56:03.210232 16355 raft_consensus.cc:348] T 6dce595b948945bfb2a381bd677e070d P d13a1a0caf8c4cd88caa7c1a3d210ee3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eb10e5e19224473a7aa433f1a863fc5" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 39023 } } peers { permanent_uuid: "12ab58de0d504260b8ea8ee564d02369" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 36347 } } peers { permanent_uuid: "d13a1a0caf8c4cd88caa7c1a3d210ee3" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 43171 } }
I20260811 18:56:03.210497 16355 raft_consensus.cc:374] T 6dce595b948945bfb2a381bd677e070d P d13a1a0caf8c4cd88caa7c1a3d210ee3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:56:03.210552 16355 raft_consensus.cc:729] T 6dce595b948945bfb2a381bd677e070d P d13a1a0caf8c4cd88caa7c1a3d210ee3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d13a1a0caf8c4cd88caa7c1a3d210ee3, State: Initialized, Role: FOLLOWER
I20260811 18:56:03.210611 16356 ts_tablet_manager.cc:1397] T 6dce595b948945bfb2a381bd677e070d P 6eb10e5e19224473a7aa433f1a863fc5: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 18:56:03.210968 16355 consensus_queue.cc:260] T 6dce595b948945bfb2a381bd677e070d P d13a1a0caf8c4cd88caa7c1a3d210ee3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eb10e5e19224473a7aa433f1a863fc5" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 39023 } } peers { permanent_uuid: "12ab58de0d504260b8ea8ee564d02369" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 36347 } } peers { permanent_uuid: "d13a1a0caf8c4cd88caa7c1a3d210ee3" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 43171 } }
I20260811 18:56:03.211606 15840 heartbeater.cc:503] Master 127.27.180.190:35581 was elected leader, sending a full tablet report...
I20260811 18:56:03.211874 16355 ts_tablet_manager.cc:1428] T 6dce595b948945bfb2a381bd677e070d P d13a1a0caf8c4cd88caa7c1a3d210ee3: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260811 18:56:03.214301 16356 raft_consensus.cc:348] T 6dce595b948945bfb2a381bd677e070d P 6eb10e5e19224473a7aa433f1a863fc5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eb10e5e19224473a7aa433f1a863fc5" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 39023 } } peers { permanent_uuid: "12ab58de0d504260b8ea8ee564d02369" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 36347 } } peers { permanent_uuid: "d13a1a0caf8c4cd88caa7c1a3d210ee3" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 43171 } }
I20260811 18:56:03.214568 16356 raft_consensus.cc:374] T 6dce595b948945bfb2a381bd677e070d P 6eb10e5e19224473a7aa433f1a863fc5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:56:03.214612 16356 raft_consensus.cc:729] T 6dce595b948945bfb2a381bd677e070d P 6eb10e5e19224473a7aa433f1a863fc5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6eb10e5e19224473a7aa433f1a863fc5, State: Initialized, Role: FOLLOWER
I20260811 18:56:03.214516 16357 raft_consensus.cc:348] T 6dce595b948945bfb2a381bd677e070d P 12ab58de0d504260b8ea8ee564d02369 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eb10e5e19224473a7aa433f1a863fc5" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 39023 } } peers { permanent_uuid: "12ab58de0d504260b8ea8ee564d02369" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 36347 } } peers { permanent_uuid: "d13a1a0caf8c4cd88caa7c1a3d210ee3" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 43171 } }
I20260811 18:56:03.214731 16357 raft_consensus.cc:374] T 6dce595b948945bfb2a381bd677e070d P 12ab58de0d504260b8ea8ee564d02369 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:56:03.214771 16357 raft_consensus.cc:729] T 6dce595b948945bfb2a381bd677e070d P 12ab58de0d504260b8ea8ee564d02369 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12ab58de0d504260b8ea8ee564d02369, State: Initialized, Role: FOLLOWER
I20260811 18:56:03.215034 16356 consensus_queue.cc:260] T 6dce595b948945bfb2a381bd677e070d P 6eb10e5e19224473a7aa433f1a863fc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eb10e5e19224473a7aa433f1a863fc5" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 39023 } } peers { permanent_uuid: "12ab58de0d504260b8ea8ee564d02369" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 36347 } } peers { permanent_uuid: "d13a1a0caf8c4cd88caa7c1a3d210ee3" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 43171 } }
I20260811 18:56:03.215111 16357 consensus_queue.cc:260] T 6dce595b948945bfb2a381bd677e070d P 12ab58de0d504260b8ea8ee564d02369 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eb10e5e19224473a7aa433f1a863fc5" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 39023 } } peers { permanent_uuid: "12ab58de0d504260b8ea8ee564d02369" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 36347 } } peers { permanent_uuid: "d13a1a0caf8c4cd88caa7c1a3d210ee3" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 43171 } }
I20260811 18:56:03.215729 16078 heartbeater.cc:503] Master 127.27.180.190:35581 was elected leader, sending a full tablet report...
I20260811 18:56:03.216008 16357 ts_tablet_manager.cc:1428] T 6dce595b948945bfb2a381bd677e070d P 12ab58de0d504260b8ea8ee564d02369: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260811 18:56:03.216008 16356 ts_tablet_manager.cc:1428] T 6dce595b948945bfb2a381bd677e070d P 6eb10e5e19224473a7aa433f1a863fc5: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260811 18:56:03.216336 16330 heartbeater.cc:503] Master 127.27.180.190:35581 was elected leader, sending a full tablet report...
I20260811 18:56:03.338657 16371 raft_consensus.cc:482] T 6dce595b948945bfb2a381bd677e070d P 6eb10e5e19224473a7aa433f1a863fc5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:56:03.338871 16371 raft_consensus.cc:504] T 6dce595b948945bfb2a381bd677e070d P 6eb10e5e19224473a7aa433f1a863fc5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eb10e5e19224473a7aa433f1a863fc5" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 39023 } } peers { permanent_uuid: "12ab58de0d504260b8ea8ee564d02369" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 36347 } } peers { permanent_uuid: "d13a1a0caf8c4cd88caa7c1a3d210ee3" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 43171 } }
I20260811 18:56:03.340088 16371 leader_election.cc:288] T 6dce595b948945bfb2a381bd677e070d P 6eb10e5e19224473a7aa433f1a863fc5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12ab58de0d504260b8ea8ee564d02369 (127.27.180.130:36347), d13a1a0caf8c4cd88caa7c1a3d210ee3 (127.27.180.129:43171)
I20260811 18:56:03.344763 16354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:56:03.340838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.131:41555 (local address 127.27.180.129:43171)
0811 18:56:03.340962 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0811 18:56:03.340966 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:56:03.341123 (+ 157us) server_negotiation.cc:407] Connection header received
0811 18:56:03.341258 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:56:03.341261 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:56:03.341307 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:56:03.341384 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:56:03.341601 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:56:03.342282 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:56:03.343608 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:56:03.344197 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:56:03.344295 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:56:03.344340 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0811 18:56:03.344584 (+ 244us) server_negotiation.cc:299] Negotiation successful
0811 18:56:03.344650 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 18:56:03.344763 16351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:56:03.340466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.131:49559 (local address 127.27.180.130:36347)
0811 18:56:03.340589 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0811 18:56:03.340593 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:56:03.340608 (+ 15us) server_negotiation.cc:407] Connection header received
0811 18:56:03.340782 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:56:03.340785 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:56:03.340841 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:56:03.340912 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:56:03.341256 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:56:03.342022 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:56:03.343504 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:56:03.344121 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:56:03.344248 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:56:03.344341 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 18:56:03.344554 (+ 213us) server_negotiation.cc:299] Negotiation successful
0811 18:56:03.344631 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 18:56:03.344769 16333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:56:03.340403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.130:36347 (local address 127.27.180.131:49559)
0811 18:56:03.340529 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0811 18:56:03.340544 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 18:56:03.340636 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:56:03.340989 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:56:03.340992 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:56:03.341012 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:56:03.341158 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:56:03.341165 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:56:03.342137 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:56:03.342140 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:56:03.343358 (+ 1218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:56:03.343370 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:56:03.344291 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:56:03.344295 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:56:03.344443 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:56:03.344459 (+ 16us) client_negotiation.cc:769] Sending connection context
0811 18:56:03.344533 (+ 74us) client_negotiation.cc:240] Negotiation successful
0811 18:56:03.344616 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41,"mutex_wait_us":22}
I20260811 18:56:03.344794 16373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:56:03.340667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.129:43171 (local address 127.27.180.131:41555)
0811 18:56:03.341013 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0811 18:56:03.341025 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 18:56:03.341159 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:56:03.341396 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:56:03.341398 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:56:03.341414 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:56:03.341504 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:56:03.341508 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:56:03.342385 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:56:03.342388 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:56:03.343490 (+ 1102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:56:03.343499 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:56:03.344301 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:56:03.344304 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:56:03.344417 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:56:03.344451 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 18:56:03.344571 (+ 120us) client_negotiation.cc:240] Negotiation successful
0811 18:56:03.344641 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"mutex_wait_us":5,"thread_start_us":127,"threads_started":1}
I20260811 18:56:03.345466 16001 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dce595b948945bfb2a381bd677e070d" candidate_uuid: "6eb10e5e19224473a7aa433f1a863fc5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12ab58de0d504260b8ea8ee564d02369" is_pre_election: true
I20260811 18:56:03.345547 15756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dce595b948945bfb2a381bd677e070d" candidate_uuid: "6eb10e5e19224473a7aa433f1a863fc5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d13a1a0caf8c4cd88caa7c1a3d210ee3" is_pre_election: true
I20260811 18:56:03.345829 16001 raft_consensus.cc:2454] T 6dce595b948945bfb2a381bd677e070d P 12ab58de0d504260b8ea8ee564d02369 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6eb10e5e19224473a7aa433f1a863fc5 in term 0.
I20260811 18:56:03.345827 15756 raft_consensus.cc:2454] T 6dce595b948945bfb2a381bd677e070d P d13a1a0caf8c4cd88caa7c1a3d210ee3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6eb10e5e19224473a7aa433f1a863fc5 in term 0.
I20260811 18:56:03.346309 16116 leader_election.cc:302] T 6dce595b948945bfb2a381bd677e070d P 6eb10e5e19224473a7aa433f1a863fc5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6eb10e5e19224473a7aa433f1a863fc5, d13a1a0caf8c4cd88caa7c1a3d210ee3; no voters:
I20260811 18:56:03.346585 16371 raft_consensus.cc:2781] T 6dce595b948945bfb2a381bd677e070d P 6eb10e5e19224473a7aa433f1a863fc5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:56:03.346664 16371 raft_consensus.cc:482] T 6dce595b948945bfb2a381bd677e070d P 6eb10e5e19224473a7aa433f1a863fc5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:56:03.346695 16371 raft_consensus.cc:3037] T 6dce595b948945bfb2a381bd677e070d P 6eb10e5e19224473a7aa433f1a863fc5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:56:03.347720 16371 raft_consensus.cc:504] T 6dce595b948945bfb2a381bd677e070d P 6eb10e5e19224473a7aa433f1a863fc5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eb10e5e19224473a7aa433f1a863fc5" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 39023 } } peers { permanent_uuid: "12ab58de0d504260b8ea8ee564d02369" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 36347 } } peers { permanent_uuid: "d13a1a0caf8c4cd88caa7c1a3d210ee3" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 43171 } }
I20260811 18:56:03.348042 16371 leader_election.cc:288] T 6dce595b948945bfb2a381bd677e070d P 6eb10e5e19224473a7aa433f1a863fc5 [CANDIDATE]: Term 1 election: Requested vote from peers 12ab58de0d504260b8ea8ee564d02369 (127.27.180.130:36347), d13a1a0caf8c4cd88caa7c1a3d210ee3 (127.27.180.129:43171)
I20260811 18:56:03.348408 16001 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dce595b948945bfb2a381bd677e070d" candidate_uuid: "6eb10e5e19224473a7aa433f1a863fc5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12ab58de0d504260b8ea8ee564d02369"
I20260811 18:56:03.348418 15756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dce595b948945bfb2a381bd677e070d" candidate_uuid: "6eb10e5e19224473a7aa433f1a863fc5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d13a1a0caf8c4cd88caa7c1a3d210ee3"
I20260811 18:56:03.348512 16001 raft_consensus.cc:3037] T 6dce595b948945bfb2a381bd677e070d P 12ab58de0d504260b8ea8ee564d02369 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:56:03.348531 15756 raft_consensus.cc:3037] T 6dce595b948945bfb2a381bd677e070d P d13a1a0caf8c4cd88caa7c1a3d210ee3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:56:03.349575 16001 raft_consensus.cc:2454] T 6dce595b948945bfb2a381bd677e070d P 12ab58de0d504260b8ea8ee564d02369 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6eb10e5e19224473a7aa433f1a863fc5 in term 1.
I20260811 18:56:03.349597 15756 raft_consensus.cc:2454] T 6dce595b948945bfb2a381bd677e070d P d13a1a0caf8c4cd88caa7c1a3d210ee3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6eb10e5e19224473a7aa433f1a863fc5 in term 1.
I20260811 18:56:03.349843 16116 leader_election.cc:302] T 6dce595b948945bfb2a381bd677e070d P 6eb10e5e19224473a7aa433f1a863fc5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6eb10e5e19224473a7aa433f1a863fc5, d13a1a0caf8c4cd88caa7c1a3d210ee3; no voters:
I20260811 18:56:03.349990 16371 raft_consensus.cc:2781] T 6dce595b948945bfb2a381bd677e070d P 6eb10e5e19224473a7aa433f1a863fc5 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:56:03.350207 16371 raft_consensus.cc:686] T 6dce595b948945bfb2a381bd677e070d P 6eb10e5e19224473a7aa433f1a863fc5 [term 1 LEADER]: Becoming Leader. State: Replica: 6eb10e5e19224473a7aa433f1a863fc5, State: Running, Role: LEADER
I20260811 18:56:03.350553 16371 consensus_queue.cc:237] T 6dce595b948945bfb2a381bd677e070d P 6eb10e5e19224473a7aa433f1a863fc5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eb10e5e19224473a7aa433f1a863fc5" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 39023 } } peers { permanent_uuid: "12ab58de0d504260b8ea8ee564d02369" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 36347 } } peers { permanent_uuid: "d13a1a0caf8c4cd88caa7c1a3d210ee3" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 43171 } }
I20260811 18:56:03.353688 15497 catalog_manager.cc:5495] T 6dce595b948945bfb2a381bd677e070d P 6eb10e5e19224473a7aa433f1a863fc5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6eb10e5e19224473a7aa433f1a863fc5 (127.27.180.131). New cstate: current_term: 1 leader_uuid: "6eb10e5e19224473a7aa433f1a863fc5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eb10e5e19224473a7aa433f1a863fc5" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 39023 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "12ab58de0d504260b8ea8ee564d02369" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 36347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d13a1a0caf8c4cd88caa7c1a3d210ee3" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 43171 } health_report { overall_health: UNKNOWN } } }
I20260811 18:56:03.380498 16350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:56:03.377307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56178 (local address 127.27.180.131:39023)
0811 18:56:03.377418 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0811 18:56:03.377421 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:56:03.377433 (+ 12us) server_negotiation.cc:407] Connection header received
0811 18:56:03.377501 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:56:03.377504 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:56:03.377558 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:56:03.377648 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:56:03.377861 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:56:03.378381 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:56:03.379248 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:56:03.379467 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:56:03.379544 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:56:03.379776 (+ 232us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:56:03.380164 (+ 388us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:56:03.380290 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 18:56:03.380316 (+ 26us) server_negotiation.cc:299] Negotiation successful
0811 18:56:03.380368 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 18:56:03.386353 15756 raft_consensus.cc:1261] T 6dce595b948945bfb2a381bd677e070d P d13a1a0caf8c4cd88caa7c1a3d210ee3 [term 1 FOLLOWER]: Refusing update from remote peer 6eb10e5e19224473a7aa433f1a863fc5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:56:03.386337 16001 raft_consensus.cc:1261] T 6dce595b948945bfb2a381bd677e070d P 12ab58de0d504260b8ea8ee564d02369 [term 1 FOLLOWER]: Refusing update from remote peer 6eb10e5e19224473a7aa433f1a863fc5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:56:03.387039 16371 consensus_queue.cc:1035] T 6dce595b948945bfb2a381bd677e070d P 6eb10e5e19224473a7aa433f1a863fc5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d13a1a0caf8c4cd88caa7c1a3d210ee3" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 43171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:56:03.387240 16375 consensus_queue.cc:1035] T 6dce595b948945bfb2a381bd677e070d P 6eb10e5e19224473a7aa433f1a863fc5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12ab58de0d504260b8ea8ee564d02369" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 36347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:56:03.394981 16381 mvcc.cc:205] Tried to move back new op lower bound from 7317399811622645760 to 7317399811486408704. Current Snapshot: MvccSnapshot[applied={T|T < 7317399811622645760}]
I20260811 18:56:03.404637 15496 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45250:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:56:03.404831 15496 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45250:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:56:03.407449 15496 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5390ce0a0c714095b34a0f12eddbd5fb: Sending DeleteTablet for 3 replicas of tablet 6dce595b948945bfb2a381bd677e070d
I20260811 18:56:03.408174 16203 tablet_service.cc:1514] Processing DeleteTablet for tablet 6dce595b948945bfb2a381bd677e070d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:56:03 UTC) from {username='kudu', principal='kudu/127.27.180.190@KRBTEST.COM'} at 127.0.0.1:56170
I20260811 18:56:03.408213 15957 tablet_service.cc:1514] Processing DeleteTablet for tablet 6dce595b948945bfb2a381bd677e070d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:56:03 UTC) from {username='kudu', principal='kudu/127.27.180.190@KRBTEST.COM'} at 127.0.0.1:43492
I20260811 18:56:03.408955 16393 tablet_replica.cc:330] T 6dce595b948945bfb2a381bd677e070d P 12ab58de0d504260b8ea8ee564d02369: stopping tablet replica
I20260811 18:56:03.409310 16393 raft_consensus.cc:2229] T 6dce595b948945bfb2a381bd677e070d P 12ab58de0d504260b8ea8ee564d02369 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:56:03.409314 15720 tablet_service.cc:1514] Processing DeleteTablet for tablet 6dce595b948945bfb2a381bd677e070d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:56:03 UTC) from {username='kudu', principal='kudu/127.27.180.190@KRBTEST.COM'} at 127.0.0.1:44608
I20260811 18:56:03.409610 16393 raft_consensus.cc:2258] T 6dce595b948945bfb2a381bd677e070d P 12ab58de0d504260b8ea8ee564d02369 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:56:03.409915 16392 tablet_replica.cc:330] T 6dce595b948945bfb2a381bd677e070d P 6eb10e5e19224473a7aa433f1a863fc5: stopping tablet replica
I20260811 18:56:03.410182 16392 raft_consensus.cc:2229] T 6dce595b948945bfb2a381bd677e070d P 6eb10e5e19224473a7aa433f1a863fc5 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:56:03.410439 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 15602
I20260811 18:56:03.416783 16393 ts_tablet_manager.cc:1905] T 6dce595b948945bfb2a381bd677e070d P 12ab58de0d504260b8ea8ee564d02369: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:56:03.419565 16393 ts_tablet_manager.cc:1918] T 6dce595b948945bfb2a381bd677e070d P 12ab58de0d504260b8ea8ee564d02369: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:56:03.419656 16393 log.cc:1196] T 6dce595b948945bfb2a381bd677e070d P 12ab58de0d504260b8ea8ee564d02369: Deleting WAL directory at /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474496157106-28370-0/minicluster-data/ts-1/wal/wals/6dce595b948945bfb2a381bd677e070d
I20260811 18:56:03.419946 16393 ts_tablet_manager.cc:1939] T 6dce595b948945bfb2a381bd677e070d P 12ab58de0d504260b8ea8ee564d02369: Deleting consensus metadata
I20260811 18:56:03.421113 16392 raft_consensus.cc:2258] T 6dce595b948945bfb2a381bd677e070d P 6eb10e5e19224473a7aa433f1a863fc5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:56:03.421731 15466 catalog_manager.cc:4841] TS 12ab58de0d504260b8ea8ee564d02369 (127.27.180.130:36347): tablet 6dce595b948945bfb2a381bd677e070d (table test-table [id=3f618d12b2024987a05406136bc21153]) successfully deleted
I20260811 18:56:03.422610 16392 ts_tablet_manager.cc:1905] T 6dce595b948945bfb2a381bd677e070d P 6eb10e5e19224473a7aa433f1a863fc5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:56:03.425344 16392 ts_tablet_manager.cc:1918] T 6dce595b948945bfb2a381bd677e070d P 6eb10e5e19224473a7aa433f1a863fc5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:56:03.425444 16392 log.cc:1196] T 6dce595b948945bfb2a381bd677e070d P 6eb10e5e19224473a7aa433f1a863fc5: Deleting WAL directory at /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786474496157106-28370-0/minicluster-data/ts-2/wal/wals/6dce595b948945bfb2a381bd677e070d
I20260811 18:56:03.425760 16392 ts_tablet_manager.cc:1939] T 6dce595b948945bfb2a381bd677e070d P 6eb10e5e19224473a7aa433f1a863fc5: Deleting consensus metadata
I20260811 18:56:03.426982 15468 catalog_manager.cc:4841] TS 6eb10e5e19224473a7aa433f1a863fc5 (127.27.180.131:39023): tablet 6dce595b948945bfb2a381bd677e070d (table test-table [id=3f618d12b2024987a05406136bc21153]) successfully deleted
W20260811 18:56:03.429617 15466 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.27.180.129:43171 (error 108)
I20260811 18:56:03.430430 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 15846
I20260811 18:56:03.430739 16349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:56:03.430421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.129:43171 (local address 127.0.0.1:44610)
0811 18:56:03.430571 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0811 18:56:03.430649 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.129:43171: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260811 18:56:03.438051 15466 catalog_manager.cc:4584] TS d13a1a0caf8c4cd88caa7c1a3d210ee3 (127.27.180.129:43171): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6dce595b948945bfb2a381bd677e070d: Network error: Client connection negotiation failed: client connection to 127.27.180.129:43171: connect: Connection refused (error 111)
I20260811 18:56:03.466250 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 16090
I20260811 18:56:03.485281 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 15431
2026-08-11T18:56:03Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2609 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:56:03 dist-test-slave-2j7r krb5kdc[16400](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:56:03 dist-test-slave-2j7r krb5kdc[16400](info): set up 2 sockets
Aug 11 18:56:03 dist-test-slave-2j7r krb5kdc[16400](info): commencing operation
krb5kdc: starting...
W20260811 18:56:04.695287 28370 mini_kdc.cc:121] Time spent starting KDC: real 1.173s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/krb5kdc/test-user.keytab.
Aug 11 18:56:04 dist-test-slave-2j7r krb5kdc[16400](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474564, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:56:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:56:04Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.27.180.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.27.180.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.27.180.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/krb5kdc/rangeradmin_127.27.180.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.27.180.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/krb5kdc/rangeradmin_127.27.180.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.27.180.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.27.180.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.27.180.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/krb5kdc/rangerlookup_127.27.180.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.27.180.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/krb5kdc/rangerlookup_127.27.180.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.27.180.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.27.180.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.27.180.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/krb5kdc/HTTP_127.27.180.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.27.180.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/krb5kdc/HTTP_127.27.180.148@KRBTEST.COM.keytab.
I20260811 18:56:04.855158 28370 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-tasktpMgJ1/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/minicluster-data/postgres -l logfile start
2026-08-11 18:56:07.194 UTC [16462] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-11 18:56:07.194 UTC [16462] LOG: listening on IPv4 address "127.27.180.148", port 34335
2026-08-11 18:56:07.197 UTC [16462] LOG: listening on Unix socket "/tmp/.s.PGSQL.34335"
2026-08-11 18:56:07.217 UTC [16465] LOG: database system was shut down at 2026-08-11 18:56:05 UTC
2026-08-11 18:56:07.221 UTC [16462] LOG: database system is ready to accept connections
I20260811 18:56:08.319145 28370 mini_postgres.cc:96] Postgres bound to 34335
2026-08-11 18:56:08.326 UTC [16479] FATAL: database "slave" does not exist
127.27.180.148:34335 - accepting connections
I20260811 18:56:08.326990 28370 mini_ranger.cc:161] Starting Ranger...
I20260811 18:56:08.341508 28370 mini_ranger.cc:84] Created miniranger Postgres user
I20260811 18:56:08.768541 28370 mini_ranger.cc:87] Created ranger Postgres database
I20260811 18:56:08.768639 28370 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/minicluster-data/ranger-admin
2026-08-11 18:56:09,061 [I] DB FLAVOR :POSTGRES
2026-08-11 18:56:09,061 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 18:56:09,062 [I] Checking connection..
2026-08-11 18:56:09,062 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpMgJ1/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktpMgJ1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.180.148:34335/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-11 18:56:09,292 [I] Checking connection passed.
2026-08-11 18:56:09,292 [I] --------- Verifying version history table ---------
2026-08-11 18:56:09,292 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpMgJ1/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktpMgJ1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.180.148:34335/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 18:56:09,532 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 18:56:09,533 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpMgJ1/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktpMgJ1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.180.148:34335/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 18:56:09,767 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 18:56:09,767 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-11 18:56:09,767 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpMgJ1/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktpMgJ1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.180.148:34335/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasktpMgJ1/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-11 18:56:10.024 UTC [16589] WARNING: there is no transaction in progress
2026-08-11 18:56:10,031 [I] create_dbversion_catalog.sql file imported successfully
2026-08-11 18:56:10,031 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpMgJ1/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktpMgJ1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.180.148:34335/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 18:56:10,271 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-11 18:56:10,271 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 18:56:10,271 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpMgJ1/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktpMgJ1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.180.148:34335/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-11 18:56:10,529 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpMgJ1/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktpMgJ1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.180.148:34335/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-11 18:56:10,769 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpMgJ1/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktpMgJ1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.180.148:34335/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-2j7r.c.gcp-upstream.internal','N') ;"
2026-08-11 18:56:11,003 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpMgJ1/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktpMgJ1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.180.148:34335/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 18:56:11,243 [I] Table x_portal_user does not exist in database ranger
2026-08-11 18:56:11,244 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpMgJ1/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktpMgJ1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.180.148:34335/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 18:56:11,489 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-11 18:56:11,490 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpMgJ1/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktpMgJ1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.180.148:34335/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 18:56:11,724 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpMgJ1/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktpMgJ1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.180.148:34335/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 18:56:11,984 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-11 18:56:11,984 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpMgJ1/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktpMgJ1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.180.148:34335/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasktpMgJ1/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-11 18:56:12.895 UTC [16770] WARNING: there is no transaction in progress
2026-08-11 18:56:12.911 UTC [16770] WARNING: there is no transaction in progress
2026-08-11 18:56:12.926 UTC [16770] WARNING: there is no transaction in progress
2026-08-11 18:56:12.941 UTC [16770] WARNING: there is no transaction in progress
2026-08-11 18:56:12.958 UTC [16770] WARNING: there is no transaction in progress
2026-08-11 18:56:12.976 UTC [16770] WARNING: there is no transaction in progress
2026-08-11 18:56:13.042 UTC [16770] WARNING: there is no transaction in progress
2026-08-11 18:56:13.049 UTC [16770] WARNING: there is no transaction in progress
2026-08-11 18:56:13.063 UTC [16770] WARNING: there is no transaction in progress
2026-08-11 18:56:13.073 UTC [16770] WARNING: there is no transaction in progress
2026-08-11 18:56:13.082 UTC [16770] WARNING: there is no transaction in progress
2026-08-11 18:56:13.095 UTC [16770] WARNING: there is no transaction in progress
2026-08-11 18:56:13.104 UTC [16770] WARNING: there is no transaction in progress
2026-08-11 18:56:13.110 UTC [16770] WARNING: there is no transaction in progress
2026-08-11 18:56:13.117 UTC [16770] WARNING: there is no transaction in progress
2026-08-11 18:56:13.124 UTC [16770] WARNING: there is no transaction in progress
2026-08-11 18:56:13,729 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-11 18:56:13,730 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpMgJ1/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktpMgJ1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.180.148:34335/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-11 18:56:13,984 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-11 18:56:13,984 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpMgJ1/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktpMgJ1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.180.148:34335/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 18:56:14,237 [I] Table x_portal_user already exists in database 'ranger'
2026-08-11 18:56:14,237 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpMgJ1/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktpMgJ1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.180.148:34335/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 18:56:14,488 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-11 18:56:14,488 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpMgJ1/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktpMgJ1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.180.148:34335/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 18:56:14,744 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpMgJ1/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktpMgJ1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.180.148:34335/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 18:56:14,991 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpMgJ1/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktpMgJ1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.180.148:34335/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-2j7r.c.gcp-upstream.internal';"
2026-08-11 18:56:15,242 [I] CORE_DB_SCHEMA import status has been updated
2026-08-11 18:56:15,242 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpMgJ1/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktpMgJ1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.180.148:34335/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-11 18:56:15,500 [I] DB_PATCHES have already been applied
I20260811 18:56:15.506623 28370 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/minicluster-data/ranger-admin:/tmp/dist-test-tasktpMgJ1/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-tasktpMgJ1/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-tasktpMgJ1/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktpMgJ1/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260811 18:56:15.506685 28370 mini_ranger.cc:193] Using host: 127.27.180.148
I20260811 18:56:15.511168 28370 mini_ranger.cc:227] Ranger admin URL: http://127.27.180.148:37085
Aug 11, 2026 6:56:15 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 6:56:15 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 6:56:15 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/minicluster-data/ranger-admin/ews/webapp
Aug 11, 2026 6:56:15 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 11, 2026 6:56:15 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 11, 2026 6:56:16 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/minicluster-data/ranger-admin/ews/webapp].
Aug 11, 2026 6:56:16 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260811 18:56:16.835347 28370 mini_ranger.cc:160] Time spent starting Ranger: real 8.508s user 0.008s sys 0.003s
Aug 11, 2026 6:56:35 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 11, 2026 6:56:35 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 11, 2026 6:56:35 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 11, 2026 6:56:35 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 11, 2026 6:56:35 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 11, 2026 6:56:35 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 11, 2026 6:56:35 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 11, 2026 6:56:35 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 11, 2026 6:56:35 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 11, 2026 6:56:35 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 11, 2026 6:56:35 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 11, 2026 6:56:35 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 11, 2026 6:56:35 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 11, 2026 6:56:35 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 11, 2026 6:56:35 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 11, 2026 6:56:35 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 11, 2026 6:56:35 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 11, 2026 6:56:36 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260811 18:56:37.375411 28370 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.27.180.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.27.180.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.27.180.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/krb5kdc/rangerkms_127.27.180.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.27.180.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/krb5kdc/rangerkms_127.27.180.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.27.180.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.27.180.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.27.180.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/krb5kdc/HTTP_127.27.180.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.27.180.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/krb5kdc/HTTP_127.27.180.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 11 18:56:37 dist-test-slave-2j7r krb5kdc[16400](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474597, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260811 18:56:37.458415 28370 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260811 18:56:37.468653 28370 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260811 18:56:38.292106 28370 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260811 18:56:38.292196 28370 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/minicluster-data/ranger-kms
I20260811 18:56:38.292299 28370 mini_ranger_kms.cc:227] Using postgres at 127.27.180.148:34335
2026-08-11 18:56:38,607 [I] DB FLAVOR :POSTGRES
2026-08-11 18:56:38,607 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 18:56:38,607 [I] Checking connection
2026-08-11 18:56:38,607 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpMgJ1/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.180.148:34335/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-11 18:56:38,877 [I] connection success
2026-08-11 18:56:38,878 [I] --------- Verifying Ranger DB tables ---------
2026-08-11 18:56:38,878 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-11 18:56:38,878 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpMgJ1/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.180.148:34335/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-11 18:56:39,116 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-11 18:56:39,117 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 18:56:39,117 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-11 18:56:39,117 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpMgJ1/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.180.148:34335/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-11 18:56:39,368 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260811 18:56:39.783124 28370 mini_ranger_kms.cc:310] Created kms service
I20260811 18:56:39.962284 28370 mini_ranger_kms.cc:326] Created kudu user
I20260811 18:56:40.005295 28370 mini_ranger_kms.cc:343] Created rangerkms user
Aug 11 18:56:40 dist-test-slave-2j7r krb5kdc[16400](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474597, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.27.180.148@KRBTEST.COM
I20260811 18:56:40.376029 28370 mini_ranger_kms.cc:384] Added ranger policy
I20260811 18:56:40.376192 28370 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/minicluster-data/ranger-kms:/tmp/dist-test-tasktpMgJ1/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktpMgJ1/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-tasktpMgJ1/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260811 18:56:40.376224 28370 mini_ranger_kms.cc:240] Using host: 127.27.180.148
I20260811 18:56:40.380813 28370 mini_ranger_kms.cc:276] Ranger KMS PID: 17067
I20260811 18:56:40.380962 28370 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.27.180.148:45493
Aug 11, 2026 6:56:41 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 6:56:41 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 6:56:41 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 11, 2026 6:56:41 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 11, 2026 6:56:41 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/minicluster-data/ranger-kms/ews/webapp].
Aug 11, 2026 6:56:41 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 11 18:56:42 dist-test-slave-2j7r krb5kdc[16400](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786474602, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.27.180.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:56:43.031018 28370 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.573s user 0.001s sys 0.256s
I20260811 18:56:43.031147 28370 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260811 18:56:43.031247 28370 mini_ranger_kms.cc:401] 127.27.180.148:45493/kms/v1/keys
Aug 11 18:56:43 dist-test-slave-2j7r krb5kdc[16400](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474602, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.27.180.148@KRBTEST.COM for HTTP/127.27.180.148@KRBTEST.COM
Aug 11 18:56:43 dist-test-slave-2j7r krb5kdc[16400](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474602, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.27.180.148@KRBTEST.COM for HTTP/127.27.180.148@KRBTEST.COM
Aug 11 18:56:45 dist-test-slave-2j7r krb5kdc[16400](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474605, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.27.180.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.180.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.180.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab.
Entry for principal kudu/127.27.180.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab.
I20260811 18:56:45.831651 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.190:40539
--webserver_interface=127.27.180.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab
--principal=kudu/127.27.180.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:39305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.180.190:40539
--ranger_config_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.27.180.148:45493/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:56:45.951540 17173 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:56:45.951897 17173 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:56:45.951951 17173 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:56:45.956167 17173 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:56:45.956243 17173 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:56:45.956261 17173 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:56:45.956277 17173 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:56:45.956291 17173 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:56:45.962122 17173 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:39305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/minicluster-data/master-0/wal
--ranger_kms_url=127.27.180.148:45493/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.27.180.190:40539
--ranger_config_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab
--principal=kudu/127.27.180.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.190:40539
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.180.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17173
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:56:45.963270 17173 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:56:45.964337 17173 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:56:45.973949 17182 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:56:45.974000 17181 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:56:45.974180 17184 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:56:45.974244 17173 server_base.cc:1034] running on GCE node
I20260811 18:56:45.975028 17173 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:56:45.976478 17173 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:56:45.977686 17173 hybrid_clock.cc:648] HybridClock initialized: now 1786474605977667 us; error 34 us; skew 500 ppm
Aug 11 18:56:45 dist-test-slave-2j7r krb5kdc[16400](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474605, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:56:45.981288 17173 init.cc:377] Logged in from keytab as kudu/127.27.180.190@KRBTEST.COM (short username kudu)
I20260811 18:56:45.982223 17173 webserver.cc:460] Webserver started at http://127.27.180.190:40461/ using document root <none> and password file <none>
I20260811 18:56:45.982745 17173 fs_manager.cc:359] Metadata directory not provided
I20260811 18:56:45.982803 17173 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:56:45.983000 17173 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 18:56:45 dist-test-slave-2j7r krb5kdc[16400](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474605, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.190@KRBTEST.COM for HTTP/127.27.180.148@KRBTEST.COM
I20260811 18:56:46.033680 17173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/minicluster-data/master-0/data/instance:
uuid: "c011a3f35e7544dbbf53489d7fbd8202"
format_stamp: "Formatted at 2026-08-11 18:56:46 on dist-test-slave-2j7r"
server_key: "7bf63640aa98508c9ac439d3ee6d3a8b"
server_key_iv: "947332c5975ae242008d4fe9881e3247"
server_key_version: "kuduclusterkey@0"
I20260811 18:56:46.034395 17173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/minicluster-data/master-0/wal/instance:
uuid: "c011a3f35e7544dbbf53489d7fbd8202"
format_stamp: "Formatted at 2026-08-11 18:56:46 on dist-test-slave-2j7r"
server_key: "7bf63640aa98508c9ac439d3ee6d3a8b"
server_key_iv: "947332c5975ae242008d4fe9881e3247"
server_key_version: "kuduclusterkey@0"
I20260811 18:56:46.038569 17173 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 18:56:46.053973 17198 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:56:46.055382 17173 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260811 18:56:46.055536 17173 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/minicluster-data/master-0/wal
uuid: "c011a3f35e7544dbbf53489d7fbd8202"
format_stamp: "Formatted at 2026-08-11 18:56:46 on dist-test-slave-2j7r"
server_key: "7bf63640aa98508c9ac439d3ee6d3a8b"
server_key_iv: "947332c5975ae242008d4fe9881e3247"
server_key_version: "kuduclusterkey@0"
I20260811 18:56:46.055680 17173 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:56:46.068806 17173 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:56:46.071913 17173 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:56:46.072108 17173 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:56:46.080809 17173 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.190:40539
I20260811 18:56:46.080821 17292 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.190:40539 every 8 connection(s)
I20260811 18:56:46.081849 17173 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
I20260811 18:56:46.084633 17294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:56:46.089006 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 17173
I20260811 18:56:46.089164 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/minicluster-data/master-0/wal/instance
I20260811 18:56:46.089473 28370 external_mini_cluster.cc:1428] Setting key 51dc1c6a80b27aa6b0ee13f9c44710a1
I20260811 18:56:46.090925 17294 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c011a3f35e7544dbbf53489d7fbd8202: Bootstrap starting.
I20260811 18:56:46.093438 17294 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c011a3f35e7544dbbf53489d7fbd8202: Neither blocks nor log segments found. Creating new log.
I20260811 18:56:46.094281 17294 log.cc:824] T 00000000000000000000000000000000 P c011a3f35e7544dbbf53489d7fbd8202: Log is configured to *not* fsync() on all Append() calls
I20260811 18:56:46.096141 17294 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c011a3f35e7544dbbf53489d7fbd8202: No bootstrap required, opened a new log
Aug 11 18:56:46 dist-test-slave-2j7r krb5kdc[16400](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474605, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM
I20260811 18:56:46.100452 17294 raft_consensus.cc:348] T 00000000000000000000000000000000 P c011a3f35e7544dbbf53489d7fbd8202 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c011a3f35e7544dbbf53489d7fbd8202" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 40539 } }
I20260811 18:56:46.100668 17294 raft_consensus.cc:374] T 00000000000000000000000000000000 P c011a3f35e7544dbbf53489d7fbd8202 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:56:46.100715 17294 raft_consensus.cc:729] T 00000000000000000000000000000000 P c011a3f35e7544dbbf53489d7fbd8202 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c011a3f35e7544dbbf53489d7fbd8202, State: Initialized, Role: FOLLOWER
I20260811 18:56:46.101155 17294 consensus_queue.cc:260] T 00000000000000000000000000000000 P c011a3f35e7544dbbf53489d7fbd8202 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c011a3f35e7544dbbf53489d7fbd8202" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 40539 } }
I20260811 18:56:46.101325 17294 raft_consensus.cc:388] T 00000000000000000000000000000000 P c011a3f35e7544dbbf53489d7fbd8202 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:56:46.101404 17294 raft_consensus.cc:482] T 00000000000000000000000000000000 P c011a3f35e7544dbbf53489d7fbd8202 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:56:46.101509 17294 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c011a3f35e7544dbbf53489d7fbd8202 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:56:46.102406 17294 raft_consensus.cc:504] T 00000000000000000000000000000000 P c011a3f35e7544dbbf53489d7fbd8202 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c011a3f35e7544dbbf53489d7fbd8202" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 40539 } }
I20260811 18:56:46.102767 17294 leader_election.cc:302] T 00000000000000000000000000000000 P c011a3f35e7544dbbf53489d7fbd8202 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c011a3f35e7544dbbf53489d7fbd8202; no voters:
I20260811 18:56:46.103114 17294 leader_election.cc:288] T 00000000000000000000000000000000 P c011a3f35e7544dbbf53489d7fbd8202 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:56:46.103313 17302 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c011a3f35e7544dbbf53489d7fbd8202 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:56:46.103575 17302 raft_consensus.cc:686] T 00000000000000000000000000000000 P c011a3f35e7544dbbf53489d7fbd8202 [term 1 LEADER]: Becoming Leader. State: Replica: c011a3f35e7544dbbf53489d7fbd8202, State: Running, Role: LEADER
I20260811 18:56:46.103940 17302 consensus_queue.cc:237] T 00000000000000000000000000000000 P c011a3f35e7544dbbf53489d7fbd8202 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c011a3f35e7544dbbf53489d7fbd8202" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 40539 } }
I20260811 18:56:46.104113 17300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:56:46.090897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35904 (local address 127.27.180.190:40539)
0811 18:56:46.091371 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0811 18:56:46.091422 (+ 51us) server_negotiation.cc:399] Waiting for connection header
0811 18:56:46.091460 (+ 38us) server_negotiation.cc:407] Connection header received
0811 18:56:46.092167 (+ 707us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:56:46.092189 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:56:46.092513 (+ 324us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:56:46.092797 (+ 284us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:56:46.093733 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:56:46.094859 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:56:46.095788 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:56:46.096276 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:56:46.096445 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:56:46.099283 (+ 2838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:56:46.099288 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:56:46.099300 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:56:46.099327 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:56:46.101057 (+ 1730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:56:46.101536 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:56:46.101540 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:56:46.101544 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:56:46.101676 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:56:46.101959 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:56:46.101961 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:56:46.101962 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:56:46.102370 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:56:46.102525 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0811 18:56:46.102816 (+ 291us) server_negotiation.cc:299] Negotiation successful
0811 18:56:46.103103 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":125,"threads_started":1}
I20260811 18:56:46.104476 17294 sys_catalog.cc:564] T 00000000000000000000000000000000 P c011a3f35e7544dbbf53489d7fbd8202 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:56:46.105125 17304 sys_catalog.cc:455] T 00000000000000000000000000000000 P c011a3f35e7544dbbf53489d7fbd8202 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c011a3f35e7544dbbf53489d7fbd8202. Latest consensus state: current_term: 1 leader_uuid: "c011a3f35e7544dbbf53489d7fbd8202" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c011a3f35e7544dbbf53489d7fbd8202" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 40539 } } }
I20260811 18:56:46.105254 17304 sys_catalog.cc:458] T 00000000000000000000000000000000 P c011a3f35e7544dbbf53489d7fbd8202 [sys.catalog]: This master's current role is: LEADER
I20260811 18:56:46.105571 17303 sys_catalog.cc:455] T 00000000000000000000000000000000 P c011a3f35e7544dbbf53489d7fbd8202 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c011a3f35e7544dbbf53489d7fbd8202" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c011a3f35e7544dbbf53489d7fbd8202" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 40539 } } }
I20260811 18:56:46.105643 17303 sys_catalog.cc:458] T 00000000000000000000000000000000 P c011a3f35e7544dbbf53489d7fbd8202 [sys.catalog]: This master's current role is: LEADER
I20260811 18:56:46.107925 17294 ranger_client.cc:318] Using new properties file: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 18:56:46.108572 17308 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:56:46.114565 17308 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:56:46.122577 17308 catalog_manager.cc:1347] Generated new cluster ID: 13c69257b6444f73a30e84e02595078a
I20260811 18:56:46.122640 17308 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:56:46.140255 17308 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:56:46.141513 17308 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:56:46.150512 17308 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c011a3f35e7544dbbf53489d7fbd8202: Generated new TSK 0
I20260811 18:56:46.151242 17308 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.180.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.180.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.180.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.129.keytab.
Entry for principal kudu/127.27.180.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.129.keytab.
I20260811 18:56:46.844197 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.27.180.148:45493/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.180.129:0
--local_ip_for_outbound_sockets=127.27.180.129
--webserver_interface=127.27.180.129
--webserver_port=0
--tserver_master_addrs=127.27.180.190:40539
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.129.keytab
--principal=kudu/127.27.180.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:39305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:56:46.982831 17380 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:56:46.983089 17380 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:56:46.983130 17380 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:56:46.987784 17380 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:56:46.987912 17380 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:56:46.988015 17380 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.180.129
I20260811 18:56:46.994661 17380 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:39305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.27.180.148:45493/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.129.keytab
--principal=kudu/127.27.180.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.180.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.180.129
--webserver_port=0
--tserver_master_addrs=127.27.180.190:40539
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17380
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.180.129
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:56:46.996006 17380 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:56:46.997434 17380 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:56:47.007959 17392 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:56:47.007975 17393 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:56:47.008349 17395 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:56:47.008469 17380 server_base.cc:1034] running on GCE node
I20260811 18:56:47.008893 17380 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:56:47.009677 17380 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:56:47.010919 17380 hybrid_clock.cc:648] HybridClock initialized: now 1786474607010890 us; error 43 us; skew 500 ppm
Aug 11 18:56:47 dist-test-slave-2j7r krb5kdc[16400](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474607, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:56:47.014326 17380 init.cc:377] Logged in from keytab as kudu/127.27.180.129@KRBTEST.COM (short username kudu)
I20260811 18:56:47.015194 17380 webserver.cc:460] Webserver started at http://127.27.180.129:43411/ using document root <none> and password file <none>
I20260811 18:56:47.015726 17380 fs_manager.cc:359] Metadata directory not provided
I20260811 18:56:47.015792 17380 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:56:47.015980 17380 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 18:56:47 dist-test-slave-2j7r krb5kdc[16400](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474607, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.129@KRBTEST.COM for HTTP/127.27.180.148@KRBTEST.COM
I20260811 18:56:47.033856 17380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/minicluster-data/ts-0/data/instance:
uuid: "a3da4d49d7e14392a89a7f4c1195d668"
format_stamp: "Formatted at 2026-08-11 18:56:47 on dist-test-slave-2j7r"
server_key: "d1412d24224aea6ef4d6c91d06ba4276"
server_key_iv: "1c60af400dbd75246de801c73f91e47d"
server_key_version: "kuduclusterkey@0"
I20260811 18:56:47.034895 17380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/minicluster-data/ts-0/wal/instance:
uuid: "a3da4d49d7e14392a89a7f4c1195d668"
format_stamp: "Formatted at 2026-08-11 18:56:47 on dist-test-slave-2j7r"
server_key: "d1412d24224aea6ef4d6c91d06ba4276"
server_key_iv: "1c60af400dbd75246de801c73f91e47d"
server_key_version: "kuduclusterkey@0"
I20260811 18:56:47.042451 17380 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.004s sys 0.000s
I20260811 18:56:47.060000 17412 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:56:47.061769 17380 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 18:56:47.061898 17380 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/minicluster-data/ts-0/wal
uuid: "a3da4d49d7e14392a89a7f4c1195d668"
format_stamp: "Formatted at 2026-08-11 18:56:47 on dist-test-slave-2j7r"
server_key: "d1412d24224aea6ef4d6c91d06ba4276"
server_key_iv: "1c60af400dbd75246de801c73f91e47d"
server_key_version: "kuduclusterkey@0"
I20260811 18:56:47.062031 17380 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:56:47.079345 17380 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:56:47.082983 17380 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:56:47.083226 17380 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:56:47.083961 17380 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:56:47.085062 17380 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:56:47.085191 17380 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:56:47.085332 17380 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:56:47.085426 17380 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:56:47.102624 17380 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.129:46073
I20260811 18:56:47.103744 17380 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/minicluster-data/ts-0/data/info.pb
I20260811 18:56:47.105546 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 17380
I20260811 18:56:47.105671 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/minicluster-data/ts-0/wal/instance
I20260811 18:56:47.106014 28370 external_mini_cluster.cc:1428] Setting key fb6b070e0860c044defce3372c90685c
I20260811 18:56:47.106568 17629 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.129:46073 every 8 connection(s)
Aug 11 18:56:47 dist-test-slave-2j7r krb5kdc[16400](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474607, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.129@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.27.180.130@KRBTEST.COM; defaulting to no policy
I20260811 18:56:47.120685 17639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:56:47.108852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.129:37587 (local address 127.27.180.190:40539)
0811 18:56:47.109162 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0811 18:56:47.109171 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:56:47.109186 (+ 15us) server_negotiation.cc:407] Connection header received
0811 18:56:47.109225 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:56:47.109227 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:56:47.109271 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:56:47.109373 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:56:47.110713 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:56:47.111543 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:56:47.113203 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:56:47.113573 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:56:47.113681 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:56:47.116575 (+ 2894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:56:47.116579 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:56:47.116582 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:56:47.116611 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:56:47.118560 (+ 1949us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:56:47.119136 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:56:47.119140 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:56:47.119142 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:56:47.119226 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:56:47.119544 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:56:47.119547 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:56:47.119548 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:56:47.119761 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:56:47.119858 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 18:56:47.120380 (+ 522us) server_negotiation.cc:299] Negotiation successful
0811 18:56:47.120507 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":222,"thread_start_us":102,"threads_started":1}
I20260811 18:56:47.121665 17637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:56:47.107288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:40539 (local address 127.27.180.129:37587)
0811 18:56:47.107833 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0811 18:56:47.107888 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 18:56:47.108862 (+ 974us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:56:47.109570 (+ 708us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:56:47.109579 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:56:47.109924 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:56:47.110548 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:56:47.110566 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:56:47.111690 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:56:47.111694 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:56:47.113048 (+ 1354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:56:47.113060 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:56:47.113737 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:56:47.113741 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:56:47.113940 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:56:47.114605 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:56:47.114625 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:56:47.116388 (+ 1763us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:56:47.118690 (+ 2302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:56:47.118696 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:56:47.118710 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:56:47.119046 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:56:47.119314 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:56:47.119317 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:56:47.119319 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:56:47.119448 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:56:47.119908 (+ 460us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:56:47.119913 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:56:47.120136 (+ 223us) client_negotiation.cc:769] Sending connection context
0811 18:56:47.120336 (+ 200us) client_negotiation.cc:240] Negotiation successful
0811 18:56:47.120638 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":154,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.180.130@KRBTEST.COM" created.
I20260811 18:56:47.125181 17631 heartbeater.cc:347] Connected to a master server at 127.27.180.190:40539
I20260811 18:56:47.125433 17631 heartbeater.cc:464] Registering TS with master...
I20260811 18:56:47.126037 17631 heartbeater.cc:511] Master 127.27.180.190:40539 requested a full tablet report, sending...
I20260811 18:56:47.127816 17227 ts_manager.cc:194] Registered new tserver with Master: a3da4d49d7e14392a89a7f4c1195d668 (127.27.180.129:46073)
I20260811 18:56:47.129406 17227 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.180.129@KRBTEST.COM'} at 127.27.180.129:37587
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.180.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.130.keytab.
Entry for principal kudu/127.27.180.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.130.keytab.
I20260811 18:56:47.149889 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.27.180.148:45493/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.180.130:0
--local_ip_for_outbound_sockets=127.27.180.130
--webserver_interface=127.27.180.130
--webserver_port=0
--tserver_master_addrs=127.27.180.190:40539
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.130.keytab
--principal=kudu/127.27.180.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:39305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 18:56:47 dist-test-slave-2j7r krb5kdc[16400](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786474607, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260811 18:56:47.284174 17643 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:56:47.284422 17643 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:56:47.284479 17643 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:56:47.288554 17643 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:56:47.288611 17643 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:56:47.288686 17643 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.180.130
I20260811 18:56:47.294965 17643 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:39305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.27.180.148:45493/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.130.keytab
--principal=kudu/127.27.180.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.180.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.180.130
--webserver_port=0
--tserver_master_addrs=127.27.180.190:40539
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17643
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.180.130
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:56:47.295966 17643 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:56:47.297029 17643 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:56:47.307304 17657 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:56:47.307236 17654 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:56:47.307771 17653 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:56:47.308557 17643 server_base.cc:1034] running on GCE node
I20260811 18:56:47.308954 17643 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:56:47.309619 17643 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:56:47.317149 17643 hybrid_clock.cc:648] HybridClock initialized: now 1786474607317058 us; error 96 us; skew 500 ppm
Aug 11 18:56:47 dist-test-slave-2j7r krb5kdc[16400](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474607, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:56:47.320135 17643 init.cc:377] Logged in from keytab as kudu/127.27.180.130@KRBTEST.COM (short username kudu)
I20260811 18:56:47.321339 17643 webserver.cc:460] Webserver started at http://127.27.180.130:34791/ using document root <none> and password file <none>
I20260811 18:56:47.322003 17643 fs_manager.cc:359] Metadata directory not provided
I20260811 18:56:47.322078 17643 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:56:47.322326 17643 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 18:56:47 dist-test-slave-2j7r krb5kdc[16400](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474607, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.130@KRBTEST.COM for HTTP/127.27.180.148@KRBTEST.COM
I20260811 18:56:47.337019 17643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/minicluster-data/ts-1/data/instance:
uuid: "688367ee519146f09e27016cab0609b4"
format_stamp: "Formatted at 2026-08-11 18:56:47 on dist-test-slave-2j7r"
server_key: "2a7259dcafa63c2ed8df3c6583fa9dc2"
server_key_iv: "e1c9cf02db861890706267d7235f5bf5"
server_key_version: "kuduclusterkey@0"
I20260811 18:56:47.338006 17643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/minicluster-data/ts-1/wal/instance:
uuid: "688367ee519146f09e27016cab0609b4"
format_stamp: "Formatted at 2026-08-11 18:56:47 on dist-test-slave-2j7r"
server_key: "2a7259dcafa63c2ed8df3c6583fa9dc2"
server_key_iv: "e1c9cf02db861890706267d7235f5bf5"
server_key_version: "kuduclusterkey@0"
I20260811 18:56:47.343402 17643 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260811 18:56:47.365485 17671 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:56:47.367424 17643 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260811 18:56:47.367563 17643 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/minicluster-data/ts-1/wal
uuid: "688367ee519146f09e27016cab0609b4"
format_stamp: "Formatted at 2026-08-11 18:56:47 on dist-test-slave-2j7r"
server_key: "2a7259dcafa63c2ed8df3c6583fa9dc2"
server_key_iv: "e1c9cf02db861890706267d7235f5bf5"
server_key_version: "kuduclusterkey@0"
I20260811 18:56:47.367725 17643 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:56:47.390470 17643 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:56:47.394094 17643 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:56:47.394340 17643 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:56:47.395126 17643 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:56:47.396240 17643 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:56:47.396348 17643 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:56:47.396445 17643 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:56:47.396507 17643 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:56:47.411192 17643 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.130:46537
I20260811 18:56:47.412427 17643 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/minicluster-data/ts-1/data/info.pb
I20260811 18:56:47.414304 17883 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.130:46537 every 8 connection(s)
I20260811 18:56:47.418689 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 17643
I20260811 18:56:47.418825 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/minicluster-data/ts-1/wal/instance
I20260811 18:56:47.419160 28370 external_mini_cluster.cc:1428] Setting key 005873f6858c1604f2f5164fa9d0b7e8
Aug 11 18:56:47 dist-test-slave-2j7r krb5kdc[16400](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474607, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.130@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM
I20260811 18:56:47.432191 17639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:56:47.417757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.130:33191 (local address 127.27.180.190:40539)
0811 18:56:47.417884 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0811 18:56:47.417889 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:56:47.418674 (+ 785us) server_negotiation.cc:407] Connection header received
0811 18:56:47.420110 (+ 1436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:56:47.420113 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:56:47.420171 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:56:47.420231 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:56:47.421978 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:56:47.422778 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:56:47.424437 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:56:47.424796 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:56:47.424876 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:56:47.427948 (+ 3072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:56:47.427952 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:56:47.427955 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:56:47.427986 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:56:47.429751 (+ 1765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:56:47.430370 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:56:47.430374 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:56:47.430376 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:56:47.430462 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:56:47.430887 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:56:47.430890 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:56:47.430892 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:56:47.431113 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:56:47.431199 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 18:56:47.431898 (+ 699us) server_negotiation.cc:299] Negotiation successful
0811 18:56:47.432028 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 18:56:47.434026 17889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:56:47.416642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:40539 (local address 127.27.180.130:33191)
0811 18:56:47.418505 (+ 1863us) negotiation.cc:107] Waiting for socket to connect
0811 18:56:47.418561 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 18:56:47.419860 (+ 1299us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:56:47.420646 (+ 786us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:56:47.420655 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:56:47.421034 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:56:47.421799 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:56:47.421821 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:56:47.422939 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:56:47.422943 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:56:47.424291 (+ 1348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:56:47.424303 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:56:47.425105 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:56:47.425110 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:56:47.425317 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:56:47.425988 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:56:47.426010 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:56:47.427749 (+ 1739us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:56:47.429894 (+ 2145us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:56:47.429900 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:56:47.429915 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:56:47.430251 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:56:47.430565 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:56:47.430568 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:56:47.430570 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:56:47.430762 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:56:47.431274 (+ 512us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:56:47.431280 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:56:47.431594 (+ 314us) client_negotiation.cc:769] Sending connection context
0811 18:56:47.432639 (+ 1045us) client_negotiation.cc:240] Negotiation successful
0811 18:56:47.432979 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1648,"thread_start_us":145,"threads_started":1}
I20260811 18:56:47.435567 17886 heartbeater.cc:347] Connected to a master server at 127.27.180.190:40539
I20260811 18:56:47.435813 17886 heartbeater.cc:464] Registering TS with master...
I20260811 18:56:47.436367 17886 heartbeater.cc:511] Master 127.27.180.190:40539 requested a full tablet report, sending...
I20260811 18:56:47.437565 17227 ts_manager.cc:194] Registered new tserver with Master: 688367ee519146f09e27016cab0609b4 (127.27.180.130:46537)
I20260811 18:56:47.438313 17227 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.180.130@KRBTEST.COM'} at 127.27.180.130:33191
WARNING: no policy specified for kudu/127.27.180.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.180.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.180.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.131.keytab.
Entry for principal kudu/127.27.180.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.131.keytab.
I20260811 18:56:47.474647 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.27.180.148:45493/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.180.131:0
--local_ip_for_outbound_sockets=127.27.180.131
--webserver_interface=127.27.180.131
--webserver_port=0
--tserver_master_addrs=127.27.180.190:40539
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.131.keytab
--principal=kudu/127.27.180.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:39305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:56:47.668366 17892 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:56:47.669078 17892 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:56:47.669243 17892 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:56:47.675144 17892 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:56:47.675268 17892 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:56:47.675410 17892 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.180.131
I20260811 18:56:47.682714 17892 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:39305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.27.180.148:45493/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.131.keytab
--principal=kudu/127.27.180.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.180.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.180.131
--webserver_port=0
--tserver_master_addrs=127.27.180.190:40539
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17892
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.180.131
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:56:47.684270 17892 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:56:47.685709 17892 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:56:47.698802 17900 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:56:47.698999 17901 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:56:47.699462 17907 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:56:47.700078 17892 server_base.cc:1034] running on GCE node
I20260811 18:56:47.700644 17892 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:56:47.701468 17892 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:56:47.702641 17892 hybrid_clock.cc:648] HybridClock initialized: now 1786474607702579 us; error 76 us; skew 500 ppm
Aug 11 18:56:47 dist-test-slave-2j7r krb5kdc[16400](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474607, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:56:47.706244 17892 init.cc:377] Logged in from keytab as kudu/127.27.180.131@KRBTEST.COM (short username kudu)
I20260811 18:56:47.707226 17892 webserver.cc:460] Webserver started at http://127.27.180.131:45003/ using document root <none> and password file <none>
I20260811 18:56:47.707888 17892 fs_manager.cc:359] Metadata directory not provided
I20260811 18:56:47.707958 17892 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:56:47.708175 17892 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 18:56:47 dist-test-slave-2j7r krb5kdc[16400](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474607, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.131@KRBTEST.COM for HTTP/127.27.180.148@KRBTEST.COM
I20260811 18:56:47.735792 17892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/minicluster-data/ts-2/data/instance:
uuid: "76af69f338ae47aeb22c38872142b3a7"
format_stamp: "Formatted at 2026-08-11 18:56:47 on dist-test-slave-2j7r"
server_key: "6ed46ee8f095c2a8b89f6b6b846a22b4"
server_key_iv: "b7c8a06226479995ed4d73b63f3e958a"
server_key_version: "kuduclusterkey@0"
I20260811 18:56:47.737524 17892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/minicluster-data/ts-2/wal/instance:
uuid: "76af69f338ae47aeb22c38872142b3a7"
format_stamp: "Formatted at 2026-08-11 18:56:47 on dist-test-slave-2j7r"
server_key: "6ed46ee8f095c2a8b89f6b6b846a22b4"
server_key_iv: "b7c8a06226479995ed4d73b63f3e958a"
server_key_version: "kuduclusterkey@0"
I20260811 18:56:47.742544 17892 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260811 18:56:47.763554 17919 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:56:47.765257 17892 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260811 18:56:47.765391 17892 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/minicluster-data/ts-2/wal
uuid: "76af69f338ae47aeb22c38872142b3a7"
format_stamp: "Formatted at 2026-08-11 18:56:47 on dist-test-slave-2j7r"
server_key: "6ed46ee8f095c2a8b89f6b6b846a22b4"
server_key_iv: "b7c8a06226479995ed4d73b63f3e958a"
server_key_version: "kuduclusterkey@0"
I20260811 18:56:47.765519 17892 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
Aug 11 18:56:47 dist-test-slave-2j7r krb5kdc[16400](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474607, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.190@KRBTEST.COM for HTTP/127.27.180.148@KRBTEST.COM
I20260811 18:56:47.803881 17892 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:56:47.817657 17892 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:56:47.817893 17892 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:56:47.818679 17892 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:56:47.819849 17892 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:56:47.819923 17892 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:56:47.820003 17892 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:56:47.820042 17892 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:56:47.834355 17892 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.131:45291
I20260811 18:56:47.835573 17892 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/minicluster-data/ts-2/data/info.pb
I20260811 18:56:47.837365 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 17892
I20260811 18:56:47.837487 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/minicluster-data/ts-2/wal/instance
I20260811 18:56:47.837772 28370 external_mini_cluster.cc:1428] Setting key 44fe44c2dabfe88292b54141ae40089e
I20260811 18:56:47.846182 18129 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.131:45291 every 8 connection(s)
Aug 11 18:56:47 dist-test-slave-2j7r krb5kdc[16400](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474607, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.131@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM
I20260811 18:56:47.857570 17639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:56:47.838322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.131:36617 (local address 127.27.180.190:40539)
0811 18:56:47.838448 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0811 18:56:47.838453 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:56:47.841738 (+ 3285us) server_negotiation.cc:407] Connection header received
0811 18:56:47.843042 (+ 1304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:56:47.843046 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:56:47.843104 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:56:47.843161 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:56:47.844894 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:56:47.845708 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:56:47.847509 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:56:47.847900 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:56:47.847987 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:56:47.852898 (+ 4911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:56:47.852903 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:56:47.852905 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:56:47.852937 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:56:47.854828 (+ 1891us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:56:47.855517 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:56:47.855521 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:56:47.855523 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:56:47.855626 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:56:47.856095 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:56:47.856099 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:56:47.856101 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:56:47.856347 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:56:47.856438 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 18:56:47.857253 (+ 815us) server_negotiation.cc:299] Negotiation successful
0811 18:56:47.857389 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 18:56:47.859571 18132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:56:47.838530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:40539 (local address 127.27.180.131:36617)
0811 18:56:47.841570 (+ 3040us) negotiation.cc:107] Waiting for socket to connect
0811 18:56:47.841625 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 18:56:47.842789 (+ 1164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:56:47.843607 (+ 818us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:56:47.843617 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:56:47.844001 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:56:47.844706 (+ 705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:56:47.844729 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:56:47.845895 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:56:47.845899 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:56:47.847346 (+ 1447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:56:47.847359 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:56:47.848212 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:56:47.848217 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:56:47.848421 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:56:47.849112 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:56:47.849138 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:56:47.852691 (+ 3553us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:56:47.854992 (+ 2301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:56:47.854999 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:56:47.855014 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:56:47.855380 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:56:47.855778 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:56:47.855781 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:56:47.855784 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:56:47.855969 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:56:47.856567 (+ 598us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:56:47.856574 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:56:47.856909 (+ 335us) client_negotiation.cc:769] Sending connection context
0811 18:56:47.858065 (+ 1156us) client_negotiation.cc:240] Negotiation successful
0811 18:56:47.858413 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2813,"thread_start_us":162,"threads_started":1}
I20260811 18:56:47.861352 18130 heartbeater.cc:347] Connected to a master server at 127.27.180.190:40539
I20260811 18:56:47.861559 18130 heartbeater.cc:464] Registering TS with master...
I20260811 18:56:47.862157 18130 heartbeater.cc:511] Master 127.27.180.190:40539 requested a full tablet report, sending...
I20260811 18:56:47.865046 17227 ts_manager.cc:194] Registered new tserver with Master: 76af69f338ae47aeb22c38872142b3a7 (127.27.180.131:45291)
I20260811 18:56:47.865852 17227 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.180.131@KRBTEST.COM'} at 127.27.180.131:36617
I20260811 18:56:47.876242 28370 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 18:56:47.898797 17639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:56:47.887964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53208 (local address 127.27.180.190:40539)
0811 18:56:47.888110 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 18:56:47.888116 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:56:47.888581 (+ 465us) server_negotiation.cc:407] Connection header received
0811 18:56:47.888836 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:56:47.888841 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:56:47.888900 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:56:47.888960 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:56:47.890193 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:56:47.890977 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:56:47.892265 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:56:47.892604 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:56:47.892685 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:56:47.894707 (+ 2022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:56:47.894711 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:56:47.894714 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:56:47.894743 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:56:47.896550 (+ 1807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:56:47.897139 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:56:47.897144 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:56:47.897146 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:56:47.897260 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:56:47.897637 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:56:47.897641 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:56:47.897642 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:56:47.897887 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:56:47.898005 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 18:56:47.898441 (+ 436us) server_negotiation.cc:299] Negotiation successful
0811 18:56:47.898612 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 18:56:47.903771 17227 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53208:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:56:47.908684 17227 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:56:47.931083 18153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:56:47.919050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.129:46073 (local address 127.0.0.1:37902)
0811 18:56:47.922687 (+ 3637us) negotiation.cc:107] Waiting for socket to connect
0811 18:56:47.923455 (+ 768us) client_negotiation.cc:174] Beginning negotiation
0811 18:56:47.923589 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:56:47.924329 (+ 740us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:56:47.924335 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:56:47.924359 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:56:47.924500 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:56:47.924507 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:56:47.926629 (+ 2122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:56:47.926633 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:56:47.928044 (+ 1411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:56:47.928055 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:56:47.930471 (+ 2416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:56:47.930476 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:56:47.930616 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:56:47.930662 (+ 46us) client_negotiation.cc:769] Sending connection context
0811 18:56:47.930795 (+ 133us) client_negotiation.cc:240] Negotiation successful
0811 18:56:47.930920 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3462,"thread_start_us":105,"threads_started":1}
I20260811 18:56:47.931732 18154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:56:47.920180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37902 (local address 127.27.180.129:46073)
0811 18:56:47.922436 (+ 2256us) server_negotiation.cc:206] Beginning negotiation
0811 18:56:47.922443 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:56:47.923725 (+ 1282us) server_negotiation.cc:407] Connection header received
0811 18:56:47.923807 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:56:47.923813 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:56:47.924018 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:56:47.924152 (+ 134us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:56:47.924625 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:56:47.925809 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:56:47.929190 (+ 3381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:56:47.930025 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:56:47.930198 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:56:47.930376 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0811 18:56:47.931449 (+ 1073us) server_negotiation.cc:299] Negotiation successful
0811 18:56:47.931567 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2144,"thread_start_us":145,"threads_started":1}
I20260811 18:56:47.935719 18162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:56:47.921520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.130:46537 (local address 127.0.0.1:42958)
0811 18:56:47.929237 (+ 7717us) negotiation.cc:107] Waiting for socket to connect
0811 18:56:47.929251 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 18:56:47.929342 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:56:47.930518 (+ 1176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:56:47.930521 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:56:47.930538 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:56:47.930636 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:56:47.930641 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:56:47.932839 (+ 2198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:56:47.932842 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:56:47.933959 (+ 1117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:56:47.933970 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:56:47.935270 (+ 1300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:56:47.935274 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:56:47.935368 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:56:47.935395 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 18:56:47.935477 (+ 82us) client_negotiation.cc:240] Negotiation successful
0811 18:56:47.935551 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7650,"thread_start_us":77,"threads_started":1}
I20260811 18:56:47.935441 17482 tablet_service.cc:1467] Processing CreateTablet for tablet b8dcbd934a2649128ba46807629b53e3 (DEFAULT_TABLE table=test-table [id=b0e170d779064aa4bd75e4bde68f94c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:56:47.936334 18167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:56:47.921909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42958 (local address 127.27.180.130:46537)
0811 18:56:47.929817 (+ 7908us) server_negotiation.cc:206] Beginning negotiation
0811 18:56:47.929834 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 18:56:47.929858 (+ 24us) server_negotiation.cc:407] Connection header received
0811 18:56:47.929959 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:56:47.929965 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:56:47.930194 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:56:47.930366 (+ 172us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:56:47.931687 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:56:47.932706 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:56:47.934123 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:56:47.934929 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:56:47.935064 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:56:47.935197 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 18:56:47.936067 (+ 870us) server_negotiation.cc:299] Negotiation successful
0811 18:56:47.936174 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7784,"thread_start_us":91,"threads_started":1}
I20260811 18:56:47.936651 17482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b8dcbd934a2649128ba46807629b53e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:56:47.939141 17761 tablet_service.cc:1467] Processing CreateTablet for tablet b8dcbd934a2649128ba46807629b53e3 (DEFAULT_TABLE table=test-table [id=b0e170d779064aa4bd75e4bde68f94c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:56:47.940196 17761 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b8dcbd934a2649128ba46807629b53e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:56:47.943549 18166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:56:47.921752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.131:45291 (local address 127.0.0.1:39184)
0811 18:56:47.929499 (+ 7747us) negotiation.cc:107] Waiting for socket to connect
0811 18:56:47.929510 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 18:56:47.929575 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:56:47.931462 (+ 1887us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:56:47.931465 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:56:47.931482 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:56:47.931564 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:56:47.931569 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:56:47.935293 (+ 3724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:56:47.935296 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:56:47.941901 (+ 6605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:56:47.941912 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:56:47.943152 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:56:47.943156 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:56:47.943254 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:56:47.943279 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 18:56:47.943345 (+ 66us) client_negotiation.cc:240] Negotiation successful
0811 18:56:47.943413 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7696,"thread_start_us":77,"threads_started":1}
I20260811 18:56:47.944213 18158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:56:47.921391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39184 (local address 127.27.180.131:45291)
0811 18:56:47.930925 (+ 9534us) server_negotiation.cc:206] Beginning negotiation
0811 18:56:47.930935 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 18:56:47.930955 (+ 20us) server_negotiation.cc:407] Connection header received
0811 18:56:47.931023 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:56:47.931027 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:56:47.931191 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:56:47.931326 (+ 135us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:56:47.934110 (+ 2784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:56:47.935163 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:56:47.942058 (+ 6895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:56:47.942844 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:56:47.942964 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:56:47.943086 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 18:56:47.943973 (+ 887us) server_negotiation.cc:299] Negotiation successful
0811 18:56:47.944074 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9428,"thread_start_us":139,"threads_started":1}
I20260811 18:56:47.947162 18011 tablet_service.cc:1467] Processing CreateTablet for tablet b8dcbd934a2649128ba46807629b53e3 (DEFAULT_TABLE table=test-table [id=b0e170d779064aa4bd75e4bde68f94c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:56:47.948230 18011 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b8dcbd934a2649128ba46807629b53e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:56:47.961552 18168 tablet_bootstrap.cc:492] T b8dcbd934a2649128ba46807629b53e3 P a3da4d49d7e14392a89a7f4c1195d668: Bootstrap starting.
I20260811 18:56:47.962495 18172 tablet_bootstrap.cc:492] T b8dcbd934a2649128ba46807629b53e3 P 76af69f338ae47aeb22c38872142b3a7: Bootstrap starting.
I20260811 18:56:47.964470 18168 tablet_bootstrap.cc:654] T b8dcbd934a2649128ba46807629b53e3 P a3da4d49d7e14392a89a7f4c1195d668: Neither blocks nor log segments found. Creating new log.
I20260811 18:56:47.965214 18172 tablet_bootstrap.cc:654] T b8dcbd934a2649128ba46807629b53e3 P 76af69f338ae47aeb22c38872142b3a7: Neither blocks nor log segments found. Creating new log.
I20260811 18:56:47.965648 18168 log.cc:824] T b8dcbd934a2649128ba46807629b53e3 P a3da4d49d7e14392a89a7f4c1195d668: Log is configured to *not* fsync() on all Append() calls
I20260811 18:56:47.966042 18172 log.cc:824] T b8dcbd934a2649128ba46807629b53e3 P 76af69f338ae47aeb22c38872142b3a7: Log is configured to *not* fsync() on all Append() calls
I20260811 18:56:47.966777 18169 tablet_bootstrap.cc:492] T b8dcbd934a2649128ba46807629b53e3 P 688367ee519146f09e27016cab0609b4: Bootstrap starting.
I20260811 18:56:47.969915 18169 tablet_bootstrap.cc:654] T b8dcbd934a2649128ba46807629b53e3 P 688367ee519146f09e27016cab0609b4: Neither blocks nor log segments found. Creating new log.
I20260811 18:56:47.970718 18169 log.cc:824] T b8dcbd934a2649128ba46807629b53e3 P 688367ee519146f09e27016cab0609b4: Log is configured to *not* fsync() on all Append() calls
I20260811 18:56:47.973022 18172 tablet_bootstrap.cc:492] T b8dcbd934a2649128ba46807629b53e3 P 76af69f338ae47aeb22c38872142b3a7: No bootstrap required, opened a new log
I20260811 18:56:47.973294 18172 ts_tablet_manager.cc:1397] T b8dcbd934a2649128ba46807629b53e3 P 76af69f338ae47aeb22c38872142b3a7: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.000s
I20260811 18:56:47.976239 18168 tablet_bootstrap.cc:492] T b8dcbd934a2649128ba46807629b53e3 P a3da4d49d7e14392a89a7f4c1195d668: No bootstrap required, opened a new log
I20260811 18:56:47.976459 18168 ts_tablet_manager.cc:1397] T b8dcbd934a2649128ba46807629b53e3 P a3da4d49d7e14392a89a7f4c1195d668: Time spent bootstrapping tablet: real 0.015s user 0.006s sys 0.000s
I20260811 18:56:47.980010 18169 tablet_bootstrap.cc:492] T b8dcbd934a2649128ba46807629b53e3 P 688367ee519146f09e27016cab0609b4: No bootstrap required, opened a new log
I20260811 18:56:47.980202 18169 ts_tablet_manager.cc:1397] T b8dcbd934a2649128ba46807629b53e3 P 688367ee519146f09e27016cab0609b4: Time spent bootstrapping tablet: real 0.014s user 0.005s sys 0.000s
I20260811 18:56:47.981670 18168 raft_consensus.cc:348] T b8dcbd934a2649128ba46807629b53e3 P a3da4d49d7e14392a89a7f4c1195d668 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3da4d49d7e14392a89a7f4c1195d668" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 46073 } } peers { permanent_uuid: "76af69f338ae47aeb22c38872142b3a7" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 45291 } } peers { permanent_uuid: "688367ee519146f09e27016cab0609b4" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 46537 } }
I20260811 18:56:47.982160 18168 raft_consensus.cc:374] T b8dcbd934a2649128ba46807629b53e3 P a3da4d49d7e14392a89a7f4c1195d668 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:56:47.982342 18168 raft_consensus.cc:729] T b8dcbd934a2649128ba46807629b53e3 P a3da4d49d7e14392a89a7f4c1195d668 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3da4d49d7e14392a89a7f4c1195d668, State: Initialized, Role: FOLLOWER
I20260811 18:56:47.982947 18168 consensus_queue.cc:260] T b8dcbd934a2649128ba46807629b53e3 P a3da4d49d7e14392a89a7f4c1195d668 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3da4d49d7e14392a89a7f4c1195d668" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 46073 } } peers { permanent_uuid: "76af69f338ae47aeb22c38872142b3a7" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 45291 } } peers { permanent_uuid: "688367ee519146f09e27016cab0609b4" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 46537 } }
I20260811 18:56:47.984243 18168 ts_tablet_manager.cc:1428] T b8dcbd934a2649128ba46807629b53e3 P a3da4d49d7e14392a89a7f4c1195d668: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20260811 18:56:47.984354 18169 raft_consensus.cc:348] T b8dcbd934a2649128ba46807629b53e3 P 688367ee519146f09e27016cab0609b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3da4d49d7e14392a89a7f4c1195d668" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 46073 } } peers { permanent_uuid: "76af69f338ae47aeb22c38872142b3a7" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 45291 } } peers { permanent_uuid: "688367ee519146f09e27016cab0609b4" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 46537 } }
I20260811 18:56:47.984620 18169 raft_consensus.cc:374] T b8dcbd934a2649128ba46807629b53e3 P 688367ee519146f09e27016cab0609b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:56:47.984669 18169 raft_consensus.cc:729] T b8dcbd934a2649128ba46807629b53e3 P 688367ee519146f09e27016cab0609b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 688367ee519146f09e27016cab0609b4, State: Initialized, Role: FOLLOWER
I20260811 18:56:47.985582 18172 raft_consensus.cc:348] T b8dcbd934a2649128ba46807629b53e3 P 76af69f338ae47aeb22c38872142b3a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3da4d49d7e14392a89a7f4c1195d668" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 46073 } } peers { permanent_uuid: "76af69f338ae47aeb22c38872142b3a7" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 45291 } } peers { permanent_uuid: "688367ee519146f09e27016cab0609b4" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 46537 } }
I20260811 18:56:47.985834 18172 raft_consensus.cc:374] T b8dcbd934a2649128ba46807629b53e3 P 76af69f338ae47aeb22c38872142b3a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:56:47.985883 18172 raft_consensus.cc:729] T b8dcbd934a2649128ba46807629b53e3 P 76af69f338ae47aeb22c38872142b3a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76af69f338ae47aeb22c38872142b3a7, State: Initialized, Role: FOLLOWER
I20260811 18:56:47.986335 17631 heartbeater.cc:503] Master 127.27.180.190:40539 was elected leader, sending a full tablet report...
I20260811 18:56:47.986397 18169 consensus_queue.cc:260] T b8dcbd934a2649128ba46807629b53e3 P 688367ee519146f09e27016cab0609b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3da4d49d7e14392a89a7f4c1195d668" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 46073 } } peers { permanent_uuid: "76af69f338ae47aeb22c38872142b3a7" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 45291 } } peers { permanent_uuid: "688367ee519146f09e27016cab0609b4" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 46537 } }
I20260811 18:56:47.987402 18169 ts_tablet_manager.cc:1428] T b8dcbd934a2649128ba46807629b53e3 P 688367ee519146f09e27016cab0609b4: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260811 18:56:47.990212 18172 consensus_queue.cc:260] T b8dcbd934a2649128ba46807629b53e3 P 76af69f338ae47aeb22c38872142b3a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3da4d49d7e14392a89a7f4c1195d668" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 46073 } } peers { permanent_uuid: "76af69f338ae47aeb22c38872142b3a7" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 45291 } } peers { permanent_uuid: "688367ee519146f09e27016cab0609b4" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 46537 } }
I20260811 18:56:47.991190 18172 ts_tablet_manager.cc:1428] T b8dcbd934a2649128ba46807629b53e3 P 76af69f338ae47aeb22c38872142b3a7: Time spent starting tablet: real 0.018s user 0.005s sys 0.000s
I20260811 18:56:47.992467 17886 heartbeater.cc:503] Master 127.27.180.190:40539 was elected leader, sending a full tablet report...
I20260811 18:56:47.993641 18130 heartbeater.cc:503] Master 127.27.180.190:40539 was elected leader, sending a full tablet report...
I20260811 18:56:48.006076 18180 raft_consensus.cc:482] T b8dcbd934a2649128ba46807629b53e3 P a3da4d49d7e14392a89a7f4c1195d668 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:56:48.006278 18180 raft_consensus.cc:504] T b8dcbd934a2649128ba46807629b53e3 P a3da4d49d7e14392a89a7f4c1195d668 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3da4d49d7e14392a89a7f4c1195d668" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 46073 } } peers { permanent_uuid: "76af69f338ae47aeb22c38872142b3a7" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 45291 } } peers { permanent_uuid: "688367ee519146f09e27016cab0609b4" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 46537 } }
I20260811 18:56:48.007529 18180 leader_election.cc:288] T b8dcbd934a2649128ba46807629b53e3 P a3da4d49d7e14392a89a7f4c1195d668 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 76af69f338ae47aeb22c38872142b3a7 (127.27.180.131:45291), 688367ee519146f09e27016cab0609b4 (127.27.180.130:46537)
I20260811 18:56:48.017841 18167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:56:48.008860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.129:45073 (local address 127.27.180.130:46537)
0811 18:56:48.009006 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 18:56:48.009012 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:56:48.010235 (+ 1223us) server_negotiation.cc:407] Connection header received
0811 18:56:48.010424 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:56:48.010427 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:56:48.010472 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:56:48.010515 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:56:48.010925 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:56:48.011710 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:56:48.014207 (+ 2497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:56:48.014865 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:56:48.014954 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:56:48.015005 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0811 18:56:48.017635 (+ 2630us) server_negotiation.cc:299] Negotiation successful
0811 18:56:48.017713 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 18:56:48.018466 18184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:56:48.008081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.130:46537 (local address 127.27.180.129:45073)
0811 18:56:48.010136 (+ 2055us) negotiation.cc:107] Waiting for socket to connect
0811 18:56:48.010150 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 18:56:48.010314 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:56:48.010642 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:56:48.010645 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:56:48.010674 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:56:48.010809 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:56:48.010814 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:56:48.011845 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:56:48.011849 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:56:48.013913 (+ 2064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:56:48.013920 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:56:48.015110 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:56:48.015114 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:56:48.015209 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:56:48.015242 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 18:56:48.018213 (+ 2971us) client_negotiation.cc:240] Negotiation successful
0811 18:56:48.018298 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1983,"thread_start_us":139,"threads_started":1}
I20260811 18:56:48.019196 17768 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8dcbd934a2649128ba46807629b53e3" candidate_uuid: "a3da4d49d7e14392a89a7f4c1195d668" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "688367ee519146f09e27016cab0609b4" is_pre_election: true
I20260811 18:56:48.019515 17768 raft_consensus.cc:2454] T b8dcbd934a2649128ba46807629b53e3 P 688367ee519146f09e27016cab0609b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3da4d49d7e14392a89a7f4c1195d668 in term 0.
I20260811 18:56:48.020114 17414 leader_election.cc:302] T b8dcbd934a2649128ba46807629b53e3 P a3da4d49d7e14392a89a7f4c1195d668 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 688367ee519146f09e27016cab0609b4, a3da4d49d7e14392a89a7f4c1195d668; no voters:
I20260811 18:56:48.020429 18180 raft_consensus.cc:2781] T b8dcbd934a2649128ba46807629b53e3 P a3da4d49d7e14392a89a7f4c1195d668 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:56:48.020504 18180 raft_consensus.cc:482] T b8dcbd934a2649128ba46807629b53e3 P a3da4d49d7e14392a89a7f4c1195d668 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:56:48.020543 18180 raft_consensus.cc:3037] T b8dcbd934a2649128ba46807629b53e3 P a3da4d49d7e14392a89a7f4c1195d668 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:56:48.022079 18180 raft_consensus.cc:504] T b8dcbd934a2649128ba46807629b53e3 P a3da4d49d7e14392a89a7f4c1195d668 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3da4d49d7e14392a89a7f4c1195d668" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 46073 } } peers { permanent_uuid: "76af69f338ae47aeb22c38872142b3a7" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 45291 } } peers { permanent_uuid: "688367ee519146f09e27016cab0609b4" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 46537 } }
I20260811 18:56:48.022441 18180 leader_election.cc:288] T b8dcbd934a2649128ba46807629b53e3 P a3da4d49d7e14392a89a7f4c1195d668 [CANDIDATE]: Term 1 election: Requested vote from peers 76af69f338ae47aeb22c38872142b3a7 (127.27.180.131:45291), 688367ee519146f09e27016cab0609b4 (127.27.180.130:46537)
I20260811 18:56:48.023027 17768 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8dcbd934a2649128ba46807629b53e3" candidate_uuid: "a3da4d49d7e14392a89a7f4c1195d668" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "688367ee519146f09e27016cab0609b4"
I20260811 18:56:48.023144 17768 raft_consensus.cc:3037] T b8dcbd934a2649128ba46807629b53e3 P 688367ee519146f09e27016cab0609b4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:56:48.024411 17768 raft_consensus.cc:2454] T b8dcbd934a2649128ba46807629b53e3 P 688367ee519146f09e27016cab0609b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3da4d49d7e14392a89a7f4c1195d668 in term 1.
I20260811 18:56:48.024819 17414 leader_election.cc:302] T b8dcbd934a2649128ba46807629b53e3 P a3da4d49d7e14392a89a7f4c1195d668 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 688367ee519146f09e27016cab0609b4, a3da4d49d7e14392a89a7f4c1195d668; no voters:
I20260811 18:56:48.025038 18180 raft_consensus.cc:2781] T b8dcbd934a2649128ba46807629b53e3 P a3da4d49d7e14392a89a7f4c1195d668 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:56:48.025341 18180 raft_consensus.cc:686] T b8dcbd934a2649128ba46807629b53e3 P a3da4d49d7e14392a89a7f4c1195d668 [term 1 LEADER]: Becoming Leader. State: Replica: a3da4d49d7e14392a89a7f4c1195d668, State: Running, Role: LEADER
I20260811 18:56:48.025715 18180 consensus_queue.cc:237] T b8dcbd934a2649128ba46807629b53e3 P a3da4d49d7e14392a89a7f4c1195d668 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3da4d49d7e14392a89a7f4c1195d668" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 46073 } } peers { permanent_uuid: "76af69f338ae47aeb22c38872142b3a7" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 45291 } } peers { permanent_uuid: "688367ee519146f09e27016cab0609b4" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 46537 } }
I20260811 18:56:48.030660 17226 catalog_manager.cc:5495] T b8dcbd934a2649128ba46807629b53e3 P a3da4d49d7e14392a89a7f4c1195d668 reported cstate change: term changed from 0 to 1, leader changed from <none> to a3da4d49d7e14392a89a7f4c1195d668 (127.27.180.129). New cstate: current_term: 1 leader_uuid: "a3da4d49d7e14392a89a7f4c1195d668" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3da4d49d7e14392a89a7f4c1195d668" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 46073 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "76af69f338ae47aeb22c38872142b3a7" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 45291 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "688367ee519146f09e27016cab0609b4" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 46537 } health_report { overall_health: UNKNOWN } } }
I20260811 18:56:48.035514 18158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:56:48.009247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.129:37967 (local address 127.27.180.131:45291)
0811 18:56:48.009352 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0811 18:56:48.009356 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:56:48.013404 (+ 4048us) server_negotiation.cc:407] Connection header received
0811 18:56:48.013571 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:56:48.013574 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:56:48.013618 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:56:48.013660 (+ 42us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:56:48.015358 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:56:48.016085 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:56:48.034153 (+ 18068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:56:48.034773 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:56:48.034882 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:56:48.034936 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 18:56:48.035298 (+ 362us) server_negotiation.cc:299] Negotiation successful
0811 18:56:48.035380 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260811 18:56:48.036059 18188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:56:48.009320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.131:45291 (local address 127.27.180.129:37967)
0811 18:56:48.013316 (+ 3996us) negotiation.cc:107] Waiting for socket to connect
0811 18:56:48.013329 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:56:48.013467 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:56:48.013780 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:56:48.013782 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:56:48.013798 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:56:48.014091 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:56:48.014096 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:56:48.016254 (+ 2158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:56:48.016257 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:56:48.017484 (+ 1227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:56:48.017495 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:56:48.035040 (+ 17545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:56:48.035044 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:56:48.035156 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:56:48.035187 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 18:56:48.035824 (+ 637us) client_negotiation.cc:240] Negotiation successful
0811 18:56:48.035909 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3929,"mutex_wait_us":167,"thread_start_us":94,"threads_started":1}
I20260811 18:56:48.036900 18046 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8dcbd934a2649128ba46807629b53e3" candidate_uuid: "a3da4d49d7e14392a89a7f4c1195d668" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76af69f338ae47aeb22c38872142b3a7" is_pre_election: true
I20260811 18:56:48.037205 18045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8dcbd934a2649128ba46807629b53e3" candidate_uuid: "a3da4d49d7e14392a89a7f4c1195d668" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76af69f338ae47aeb22c38872142b3a7"
I20260811 18:56:48.038233 18046 raft_consensus.cc:2454] T b8dcbd934a2649128ba46807629b53e3 P 76af69f338ae47aeb22c38872142b3a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3da4d49d7e14392a89a7f4c1195d668 in term 0.
I20260811 18:56:48.065150 18154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:56:48.059610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37904 (local address 127.27.180.129:46073)
0811 18:56:48.059771 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0811 18:56:48.059776 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:56:48.059796 (+ 20us) server_negotiation.cc:407] Connection header received
0811 18:56:48.059889 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:56:48.059893 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:56:48.059967 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:56:48.060099 (+ 132us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:56:48.060390 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:56:48.061265 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:56:48.062603 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:56:48.062980 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:56:48.063108 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:56:48.063684 (+ 576us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:56:48.064285 (+ 601us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:56:48.064412 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0811 18:56:48.064882 (+ 470us) server_negotiation.cc:299] Negotiation successful
0811 18:56:48.064950 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 18:56:48.072834 17768 raft_consensus.cc:1261] T b8dcbd934a2649128ba46807629b53e3 P 688367ee519146f09e27016cab0609b4 [term 1 FOLLOWER]: Refusing update from remote peer a3da4d49d7e14392a89a7f4c1195d668: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:56:48.073715 18189 consensus_queue.cc:1035] T b8dcbd934a2649128ba46807629b53e3 P a3da4d49d7e14392a89a7f4c1195d668 [LEADER]: Connected to new peer: Peer: permanent_uuid: "688367ee519146f09e27016cab0609b4" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 46537 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:56:48.081604 18046 raft_consensus.cc:3037] T b8dcbd934a2649128ba46807629b53e3 P 76af69f338ae47aeb22c38872142b3a7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:56:48.083323 18046 raft_consensus.cc:1261] T b8dcbd934a2649128ba46807629b53e3 P 76af69f338ae47aeb22c38872142b3a7 [term 1 FOLLOWER]: Refusing update from remote peer a3da4d49d7e14392a89a7f4c1195d668: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:56:48.084081 18189 consensus_queue.cc:1035] T b8dcbd934a2649128ba46807629b53e3 P a3da4d49d7e14392a89a7f4c1195d668 [LEADER]: Connected to new peer: Peer: permanent_uuid: "76af69f338ae47aeb22c38872142b3a7" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 45291 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 18:56:48.089996 18131 tablet.cc:2369] T b8dcbd934a2649128ba46807629b53e3 P 76af69f338ae47aeb22c38872142b3a7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:56:48.099503 18193 mvcc.cc:205] Tried to move back new op lower bound from 7317399994656915456 to 7317399994477719552. Current Snapshot: MvccSnapshot[applied={T|T < 7317399994656915456}]
I20260811 18:56:48.101497 18198 mvcc.cc:205] Tried to move back new op lower bound from 7317399994656915456 to 7317399994477719552. Current Snapshot: MvccSnapshot[applied={T|T < 7317399994656915456}]
I20260811 18:56:48.112983 18195 mvcc.cc:205] Tried to move back new op lower bound from 7317399994656915456 to 7317399994477719552. Current Snapshot: MvccSnapshot[applied={T|T < 7317399994656915456}]
I20260811 18:56:48.125993 17226 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53208:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:56:48.126207 17226 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53208:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:56:48.129444 17226 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c011a3f35e7544dbbf53489d7fbd8202: Sending DeleteTablet for 3 replicas of tablet b8dcbd934a2649128ba46807629b53e3
I20260811 18:56:48.130568 18011 tablet_service.cc:1514] Processing DeleteTablet for tablet b8dcbd934a2649128ba46807629b53e3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:56:48 UTC) from {username='kudu', principal='kudu/127.27.180.190@KRBTEST.COM'} at 127.0.0.1:39184
I20260811 18:56:48.131711 17761 tablet_service.cc:1514] Processing DeleteTablet for tablet b8dcbd934a2649128ba46807629b53e3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:56:48 UTC) from {username='kudu', principal='kudu/127.27.180.190@KRBTEST.COM'} at 127.0.0.1:42958
I20260811 18:56:48.132670 17482 tablet_service.cc:1514] Processing DeleteTablet for tablet b8dcbd934a2649128ba46807629b53e3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:56:48 UTC) from {username='kudu', principal='kudu/127.27.180.190@KRBTEST.COM'} at 127.0.0.1:37902
I20260811 18:56:48.133319 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 17380
I20260811 18:56:48.145407 18209 tablet_replica.cc:330] T b8dcbd934a2649128ba46807629b53e3 P 76af69f338ae47aeb22c38872142b3a7: stopping tablet replica
I20260811 18:56:48.145725 18209 raft_consensus.cc:2229] T b8dcbd934a2649128ba46807629b53e3 P 76af69f338ae47aeb22c38872142b3a7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:56:48.146025 18209 raft_consensus.cc:2258] T b8dcbd934a2649128ba46807629b53e3 P 76af69f338ae47aeb22c38872142b3a7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:56:48.146355 18210 tablet_replica.cc:330] T b8dcbd934a2649128ba46807629b53e3 P 688367ee519146f09e27016cab0609b4: stopping tablet replica
I20260811 18:56:48.146598 18210 raft_consensus.cc:2229] T b8dcbd934a2649128ba46807629b53e3 P 688367ee519146f09e27016cab0609b4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:56:48.146829 18210 raft_consensus.cc:2258] T b8dcbd934a2649128ba46807629b53e3 P 688367ee519146f09e27016cab0609b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:56:48.148339 18210 ts_tablet_manager.cc:1905] T b8dcbd934a2649128ba46807629b53e3 P 688367ee519146f09e27016cab0609b4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:56:48.151409 18210 ts_tablet_manager.cc:1918] T b8dcbd934a2649128ba46807629b53e3 P 688367ee519146f09e27016cab0609b4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:56:48.151556 18210 log.cc:1196] T b8dcbd934a2649128ba46807629b53e3 P 688367ee519146f09e27016cab0609b4: Deleting WAL directory at /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/minicluster-data/ts-1/wal/wals/b8dcbd934a2649128ba46807629b53e3
I20260811 18:56:48.151994 18210 ts_tablet_manager.cc:1939] T b8dcbd934a2649128ba46807629b53e3 P 688367ee519146f09e27016cab0609b4: Deleting consensus metadata
I20260811 18:56:48.153294 17199 catalog_manager.cc:4841] TS 688367ee519146f09e27016cab0609b4 (127.27.180.130:46537): tablet b8dcbd934a2649128ba46807629b53e3 (table test-table [id=b0e170d779064aa4bd75e4bde68f94c2]) successfully deleted
I20260811 18:56:48.154758 18209 ts_tablet_manager.cc:1905] T b8dcbd934a2649128ba46807629b53e3 P 76af69f338ae47aeb22c38872142b3a7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:56:48.157589 18209 ts_tablet_manager.cc:1918] T b8dcbd934a2649128ba46807629b53e3 P 76af69f338ae47aeb22c38872142b3a7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:56:48.157711 18209 log.cc:1196] T b8dcbd934a2649128ba46807629b53e3 P 76af69f338ae47aeb22c38872142b3a7: Deleting WAL directory at /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786474496157106-28370-0/minicluster-data/ts-2/wal/wals/b8dcbd934a2649128ba46807629b53e3
I20260811 18:56:48.158401 18209 ts_tablet_manager.cc:1939] T b8dcbd934a2649128ba46807629b53e3 P 76af69f338ae47aeb22c38872142b3a7: Deleting consensus metadata
I20260811 18:56:48.159816 17206 catalog_manager.cc:4841] TS 76af69f338ae47aeb22c38872142b3a7 (127.27.180.131:45291): tablet b8dcbd934a2649128ba46807629b53e3 (table test-table [id=b0e170d779064aa4bd75e4bde68f94c2]) successfully deleted
W20260811 18:56:48.167415 17199 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.27.180.129:46073 (error 108)
I20260811 18:56:48.169243 18166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:56:48.168842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.129:46073 (local address 127.0.0.1:37908)
0811 18:56:48.169038 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0811 18:56:48.169132 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.129:46073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117}
W20260811 18:56:48.169428 17199 catalog_manager.cc:4584] TS a3da4d49d7e14392a89a7f4c1195d668 (127.27.180.129:46073): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b8dcbd934a2649128ba46807629b53e3: Network error: Client connection negotiation failed: client connection to 127.27.180.129:46073: connect: Connection refused (error 111)
I20260811 18:56:48.169816 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 17643
I20260811 18:56:48.194038 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 17892
I20260811 18:56:48.239912 18166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:56:48.239410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.129:46073 (local address 127.0.0.1:37916)
0811 18:56:48.239688 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0811 18:56:48.239777 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.129:46073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173}
I20260811 18:56:48.241122 18166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:56:48.240743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.129:46073 (local address 127.0.0.1:37926)
0811 18:56:48.240933 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0811 18:56:48.240998 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.129:46073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
I20260811 18:56:48.259917 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 17173
I20260811 18:56:48.272018 28370 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260811 18:56:48.618376 28370 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260811 18:56:48.618508 28370 mini_ranger.cc:66] Stopping Ranger...
I20260811 18:56:48.973419 28370 mini_ranger.cc:68] Stopped Ranger
2026-08-11 18:56:48.973 UTC [16462] LOG: received smart shutdown request
2026-08-11 18:56:48.977 UTC [16462] LOG: background worker "logical replication launcher" (PID 16477) exited with exit code 1
2026-08-11 18:56:48.977 UTC [16466] LOG: shutting down
2026-08-11 18:56:49.116 UTC [16462] LOG: database system is shut down
2026-08-11T18:56:49Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (45720 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:56:49 dist-test-slave-2j7r krb5kdc[18229](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:56:49 dist-test-slave-2j7r krb5kdc[18229](info): set up 2 sockets
Aug 11 18:56:49 dist-test-slave-2j7r krb5kdc[18229](info): commencing operation
krb5kdc: starting...
W20260811 18:56:50.427798 28370 mini_kdc.cc:121] Time spent starting KDC: real 1.185s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/krb5kdc/test-user.keytab.
Aug 11 18:56:50 dist-test-slave-2j7r krb5kdc[18229](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474610, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:56:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:56:50Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.27.180.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.27.180.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.27.180.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/krb5kdc/rangeradmin_127.27.180.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.27.180.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/krb5kdc/rangeradmin_127.27.180.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.27.180.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.27.180.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.27.180.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/krb5kdc/rangerlookup_127.27.180.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.27.180.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/krb5kdc/rangerlookup_127.27.180.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.27.180.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.27.180.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.27.180.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/krb5kdc/HTTP_127.27.180.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.27.180.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/krb5kdc/HTTP_127.27.180.148@KRBTEST.COM.keytab.
I20260811 18:56:50.591142 28370 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-tasktpMgJ1/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/postgres -l logfile start
2026-08-11 18:56:53.110 UTC [18283] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-11 18:56:53.110 UTC [18283] LOG: listening on IPv4 address "127.27.180.148", port 48935
2026-08-11 18:56:53.113 UTC [18283] LOG: listening on Unix socket "/tmp/.s.PGSQL.48935"
2026-08-11 18:56:53.132 UTC [18287] LOG: database system was shut down at 2026-08-11 18:56:51 UTC
2026-08-11 18:56:53.135 UTC [18283] LOG: database system is ready to accept connections
I20260811 18:56:54.247649 28370 mini_postgres.cc:96] Postgres bound to 48935
2026-08-11 18:56:54.257 UTC [18298] FATAL: database "slave" does not exist
127.27.180.148:48935 - accepting connections
I20260811 18:56:54.257669 28370 mini_ranger.cc:161] Starting Ranger...
I20260811 18:56:54.272357 28370 mini_ranger.cc:84] Created miniranger Postgres user
I20260811 18:56:54.695809 28370 mini_ranger.cc:87] Created ranger Postgres database
I20260811 18:56:54.695899 28370 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ranger-admin
2026-08-11 18:56:54,973 [I] DB FLAVOR :POSTGRES
2026-08-11 18:56:54,974 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 18:56:54,974 [I] Checking connection..
2026-08-11 18:56:54,974 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpMgJ1/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktpMgJ1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.180.148:48935/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-11 18:56:55,238 [I] Checking connection passed.
2026-08-11 18:56:55,238 [I] --------- Verifying version history table ---------
2026-08-11 18:56:55,238 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpMgJ1/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktpMgJ1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.180.148:48935/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 18:56:55,488 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 18:56:55,488 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpMgJ1/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktpMgJ1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.180.148:48935/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 18:56:55,755 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 18:56:55,755 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-11 18:56:55,755 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpMgJ1/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktpMgJ1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.180.148:48935/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasktpMgJ1/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-11 18:56:55.993 UTC [18415] WARNING: there is no transaction in progress
2026-08-11 18:56:56,002 [I] create_dbversion_catalog.sql file imported successfully
2026-08-11 18:56:56,002 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpMgJ1/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktpMgJ1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.180.148:48935/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 18:56:56,239 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-11 18:56:56,239 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 18:56:56,240 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpMgJ1/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktpMgJ1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.180.148:48935/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-11 18:56:56,492 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpMgJ1/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktpMgJ1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.180.148:48935/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-11 18:56:56,745 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpMgJ1/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktpMgJ1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.180.148:48935/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-2j7r.c.gcp-upstream.internal','N') ;"
2026-08-11 18:56:56,985 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpMgJ1/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktpMgJ1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.180.148:48935/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 18:56:57,250 [I] Table x_portal_user does not exist in database ranger
2026-08-11 18:56:57,251 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpMgJ1/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktpMgJ1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.180.148:48935/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 18:56:57,509 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-11 18:56:57,509 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpMgJ1/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktpMgJ1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.180.148:48935/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 18:56:57,744 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpMgJ1/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktpMgJ1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.180.148:48935/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 18:56:57,974 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-11 18:56:57,974 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpMgJ1/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktpMgJ1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.180.148:48935/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasktpMgJ1/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-11 18:56:58.897 UTC [18605] WARNING: there is no transaction in progress
2026-08-11 18:56:58.912 UTC [18605] WARNING: there is no transaction in progress
2026-08-11 18:56:58.927 UTC [18605] WARNING: there is no transaction in progress
2026-08-11 18:56:58.941 UTC [18605] WARNING: there is no transaction in progress
2026-08-11 18:56:58.956 UTC [18605] WARNING: there is no transaction in progress
2026-08-11 18:56:58.973 UTC [18605] WARNING: there is no transaction in progress
2026-08-11 18:56:59.035 UTC [18605] WARNING: there is no transaction in progress
2026-08-11 18:56:59.041 UTC [18605] WARNING: there is no transaction in progress
2026-08-11 18:56:59.055 UTC [18605] WARNING: there is no transaction in progress
2026-08-11 18:56:59.066 UTC [18605] WARNING: there is no transaction in progress
2026-08-11 18:56:59.076 UTC [18605] WARNING: there is no transaction in progress
2026-08-11 18:56:59.089 UTC [18605] WARNING: there is no transaction in progress
2026-08-11 18:56:59.098 UTC [18605] WARNING: there is no transaction in progress
2026-08-11 18:56:59.103 UTC [18605] WARNING: there is no transaction in progress
2026-08-11 18:56:59.109 UTC [18605] WARNING: there is no transaction in progress
2026-08-11 18:56:59.115 UTC [18605] WARNING: there is no transaction in progress
2026-08-11 18:56:59,713 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-11 18:56:59,713 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpMgJ1/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktpMgJ1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.180.148:48935/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-11 18:56:59,952 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-11 18:56:59,953 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpMgJ1/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktpMgJ1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.180.148:48935/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 18:57:00,195 [I] Table x_portal_user already exists in database 'ranger'
2026-08-11 18:57:00,196 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpMgJ1/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktpMgJ1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.180.148:48935/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 18:57:00,431 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-11 18:57:00,432 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpMgJ1/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktpMgJ1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.180.148:48935/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 18:57:00,660 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpMgJ1/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktpMgJ1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.180.148:48935/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 18:57:00,898 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpMgJ1/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktpMgJ1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.180.148:48935/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-2j7r.c.gcp-upstream.internal';"
2026-08-11 18:57:01,135 [I] CORE_DB_SCHEMA import status has been updated
2026-08-11 18:57:01,135 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpMgJ1/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktpMgJ1/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.180.148:48935/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-11 18:57:01,392 [I] DB_PATCHES have already been applied
I20260811 18:57:01.398303 28370 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ranger-admin:/tmp/dist-test-tasktpMgJ1/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-tasktpMgJ1/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-tasktpMgJ1/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktpMgJ1/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260811 18:57:01.398363 28370 mini_ranger.cc:193] Using host: 127.27.180.148
I20260811 18:57:01.402626 28370 mini_ranger.cc:227] Ranger admin URL: http://127.27.180.148:60729
Aug 11, 2026 6:57:01 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 6:57:01 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 6:57:01 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ranger-admin/ews/webapp
Aug 11, 2026 6:57:01 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 11, 2026 6:57:01 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 11, 2026 6:57:02 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ranger-admin/ews/webapp].
Aug 11, 2026 6:57:02 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260811 18:57:02.729010 28370 mini_ranger.cc:160] Time spent starting Ranger: real 8.471s user 0.001s sys 0.009s
Aug 11, 2026 6:57:22 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 11, 2026 6:57:22 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.MetricsREST
Aug 11, 2026 6:57:22 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 11, 2026 6:57:22 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 11, 2026 6:57:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 11, 2026 6:57:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 11, 2026 6:57:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 11, 2026 6:57:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 11, 2026 6:57:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 11, 2026 6:57:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 11, 2026 6:57:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 11, 2026 6:57:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 11, 2026 6:57:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 11, 2026 6:57:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 11, 2026 6:57:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 11, 2026 6:57:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 11, 2026 6:57:22 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 11, 2026 6:57:23 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260811 18:57:24.523973 28370 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.27.180.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.27.180.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.27.180.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/krb5kdc/rangerkms_127.27.180.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.27.180.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/krb5kdc/rangerkms_127.27.180.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.27.180.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.27.180.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.27.180.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/krb5kdc/HTTP_127.27.180.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.27.180.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/krb5kdc/HTTP_127.27.180.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 11 18:57:24 dist-test-slave-2j7r krb5kdc[18229](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474644, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260811 18:57:24.611354 28370 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260811 18:57:24.621556 28370 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260811 18:57:25.478099 28370 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260811 18:57:25.478173 28370 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ranger-kms
I20260811 18:57:25.478185 28370 mini_ranger_kms.cc:227] Using postgres at 127.27.180.148:48935
2026-08-11 18:57:25,640 [I] DB FLAVOR :POSTGRES
2026-08-11 18:57:25,640 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 18:57:25,640 [I] Checking connection
2026-08-11 18:57:25,641 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpMgJ1/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.180.148:48935/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-11 18:57:25,896 [I] connection success
2026-08-11 18:57:25,896 [I] --------- Verifying Ranger DB tables ---------
2026-08-11 18:57:25,897 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-11 18:57:25,897 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpMgJ1/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.180.148:48935/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-11 18:57:26,140 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-11 18:57:26,140 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 18:57:26,140 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-11 18:57:26,141 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpMgJ1/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.27.180.148:48935/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-11 18:57:26,392 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260811 18:57:26.860133 28370 mini_ranger_kms.cc:310] Created kms service
I20260811 18:57:27.056393 28370 mini_ranger_kms.cc:326] Created kudu user
I20260811 18:57:27.105078 28370 mini_ranger_kms.cc:343] Created rangerkms user
Aug 11 18:57:27 dist-test-slave-2j7r krb5kdc[18229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474644, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.27.180.148@KRBTEST.COM
I20260811 18:57:27.537616 28370 mini_ranger_kms.cc:384] Added ranger policy
I20260811 18:57:27.537886 28370 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ranger-kms:/tmp/dist-test-tasktpMgJ1/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktpMgJ1/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-tasktpMgJ1/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260811 18:57:27.537919 28370 mini_ranger_kms.cc:240] Using host: 127.27.180.148
I20260811 18:57:27.542392 28370 mini_ranger_kms.cc:276] Ranger KMS PID: 18943
I20260811 18:57:27.542465 28370 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.27.180.148:55125
Aug 11, 2026 6:57:28 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 6:57:28 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 6:57:28 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 11, 2026 6:57:28 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 11, 2026 6:57:29 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ranger-kms/ews/webapp].
Aug 11, 2026 6:57:29 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 11 18:57:29 dist-test-slave-2j7r krb5kdc[18229](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786474649, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.27.180.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:57:30.145393 28370 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.534s user 0.013s sys 0.134s
I20260811 18:57:30.145504 28370 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260811 18:57:30.145579 28370 mini_ranger_kms.cc:401] 127.27.180.148:55125/kms/v1/keys
Aug 11 18:57:30 dist-test-slave-2j7r krb5kdc[18229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474649, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.27.180.148@KRBTEST.COM for HTTP/127.27.180.148@KRBTEST.COM
Aug 11 18:57:30 dist-test-slave-2j7r krb5kdc[18229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474649, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.27.180.148@KRBTEST.COM for HTTP/127.27.180.148@KRBTEST.COM
Aug 11 18:57:32 dist-test-slave-2j7r krb5kdc[18229](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474652, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.27.180.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.180.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.180.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab.
Entry for principal kudu/127.27.180.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab.
I20260811 18:57:32.828083 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.190:33551
--webserver_interface=127.27.180.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab
--principal=kudu/127.27.180.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:44551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.180.190:33551
--ranger_config_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.27.180.148:55125/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:57:32.940467 19032 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:57:32.940712 19032 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:57:32.940750 19032 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:57:32.944785 19032 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:57:32.944856 19032 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:57:32.944881 19032 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:57:32.944906 19032 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:57:32.944972 19032 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:57:32.950711 19032 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:44551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/master-0/wal
--ranger_kms_url=127.27.180.148:55125/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.27.180.190:33551
--ranger_config_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab
--principal=kudu/127.27.180.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.190:33551
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.180.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19032
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:57:32.951725 19032 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:57:32.952766 19032 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:57:32.962049 19040 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:32.962061 19043 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:57:32.962438 19032 server_base.cc:1034] running on GCE node
W20260811 18:57:32.962587 19046 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:57:32.963145 19032 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:57:32.964272 19032 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:57:32.965456 19032 hybrid_clock.cc:648] HybridClock initialized: now 1786474652965435 us; error 43 us; skew 500 ppm
Aug 11 18:57:32 dist-test-slave-2j7r krb5kdc[18229](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474652, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:57:32.971086 19032 init.cc:377] Logged in from keytab as kudu/127.27.180.190@KRBTEST.COM (short username kudu)
I20260811 18:57:32.972011 19032 webserver.cc:460] Webserver started at http://127.27.180.190:36921/ using document root <none> and password file <none>
I20260811 18:57:32.972539 19032 fs_manager.cc:359] Metadata directory not provided
I20260811 18:57:32.972596 19032 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:57:32.972786 19032 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 18:57:32 dist-test-slave-2j7r krb5kdc[18229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474652, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.190@KRBTEST.COM for HTTP/127.27.180.148@KRBTEST.COM
I20260811 18:57:33.030102 19032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/master-0/data/instance:
uuid: "7c3b1394721845259008d27a9a7ff684"
format_stamp: "Formatted at 2026-08-11 18:57:33 on dist-test-slave-2j7r"
server_key: "9ab7aaf58fc40c1cfb6df7c0ed60cd32"
server_key_iv: "a2d3d571dfa9f746b809b3e19386c270"
server_key_version: "kuduclusterkey@0"
I20260811 18:57:33.030819 19032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/master-0/wal/instance:
uuid: "7c3b1394721845259008d27a9a7ff684"
format_stamp: "Formatted at 2026-08-11 18:57:33 on dist-test-slave-2j7r"
server_key: "9ab7aaf58fc40c1cfb6df7c0ed60cd32"
server_key_iv: "a2d3d571dfa9f746b809b3e19386c270"
server_key_version: "kuduclusterkey@0"
I20260811 18:57:33.035574 19032 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 18:57:33.049868 19055 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:33.051182 19032 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 18:57:33.051308 19032 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/master-0/wal
uuid: "7c3b1394721845259008d27a9a7ff684"
format_stamp: "Formatted at 2026-08-11 18:57:33 on dist-test-slave-2j7r"
server_key: "9ab7aaf58fc40c1cfb6df7c0ed60cd32"
server_key_iv: "a2d3d571dfa9f746b809b3e19386c270"
server_key_version: "kuduclusterkey@0"
I20260811 18:57:33.051429 19032 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:57:33.073593 19032 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:57:33.076397 19032 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:57:33.076578 19032 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:57:33.084079 19032 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.190:33551
I20260811 18:57:33.084097 19139 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.190:33551 every 8 connection(s)
I20260811 18:57:33.085122 19032 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
I20260811 18:57:33.087751 19140 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:33.093043 19140 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c3b1394721845259008d27a9a7ff684: Bootstrap starting.
I20260811 18:57:33.095340 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 19032
I20260811 18:57:33.095394 19140 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7c3b1394721845259008d27a9a7ff684: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:33.095467 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/master-0/wal/instance
I20260811 18:57:33.095758 28370 external_mini_cluster.cc:1428] Setting key b09d80dfa5ee2636d147ddeac74ae718
I20260811 18:57:33.096177 19140 log.cc:824] T 00000000000000000000000000000000 P 7c3b1394721845259008d27a9a7ff684: Log is configured to *not* fsync() on all Append() calls
I20260811 18:57:33.098678 19140 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c3b1394721845259008d27a9a7ff684: No bootstrap required, opened a new log
I20260811 18:57:33.102414 19140 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7c3b1394721845259008d27a9a7ff684 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c3b1394721845259008d27a9a7ff684" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 33551 } }
I20260811 18:57:33.102634 19140 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7c3b1394721845259008d27a9a7ff684 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:33.102705 19140 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7c3b1394721845259008d27a9a7ff684 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c3b1394721845259008d27a9a7ff684, State: Initialized, Role: FOLLOWER
I20260811 18:57:33.103168 19140 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7c3b1394721845259008d27a9a7ff684 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c3b1394721845259008d27a9a7ff684" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 33551 } }
I20260811 18:57:33.103322 19140 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7c3b1394721845259008d27a9a7ff684 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:57:33.103408 19140 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7c3b1394721845259008d27a9a7ff684 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:57:33.103508 19140 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7c3b1394721845259008d27a9a7ff684 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:33.104537 19140 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7c3b1394721845259008d27a9a7ff684 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c3b1394721845259008d27a9a7ff684" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 33551 } }
I20260811 18:57:33.104879 19140 leader_election.cc:302] T 00000000000000000000000000000000 P 7c3b1394721845259008d27a9a7ff684 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7c3b1394721845259008d27a9a7ff684; no voters:
I20260811 18:57:33.105199 19140 leader_election.cc:288] T 00000000000000000000000000000000 P 7c3b1394721845259008d27a9a7ff684 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:57:33.105271 19147 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7c3b1394721845259008d27a9a7ff684 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:33.105577 19147 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7c3b1394721845259008d27a9a7ff684 [term 1 LEADER]: Becoming Leader. State: Replica: 7c3b1394721845259008d27a9a7ff684, State: Running, Role: LEADER
I20260811 18:57:33.105917 19147 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7c3b1394721845259008d27a9a7ff684 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c3b1394721845259008d27a9a7ff684" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 33551 } }
Aug 11 18:57:33 dist-test-slave-2j7r krb5kdc[18229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474652, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM
I20260811 18:57:33.106585 19140 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7c3b1394721845259008d27a9a7ff684 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:57:33.107136 19147 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c3b1394721845259008d27a9a7ff684 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7c3b1394721845259008d27a9a7ff684" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c3b1394721845259008d27a9a7ff684" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 33551 } } }
I20260811 18:57:33.107347 19147 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c3b1394721845259008d27a9a7ff684 [sys.catalog]: This master's current role is: LEADER
I20260811 18:57:33.107594 19147 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c3b1394721845259008d27a9a7ff684 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7c3b1394721845259008d27a9a7ff684. Latest consensus state: current_term: 1 leader_uuid: "7c3b1394721845259008d27a9a7ff684" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c3b1394721845259008d27a9a7ff684" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 33551 } } }
I20260811 18:57:33.107741 19147 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c3b1394721845259008d27a9a7ff684 [sys.catalog]: This master's current role is: LEADER
I20260811 18:57:33.108040 19166 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:57:33.111249 19166 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:57:33.111857 19140 ranger_client.cc:318] Using new properties file: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 18:57:33.112548 19146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:33.096901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49204 (local address 127.27.180.190:33551)
0811 18:57:33.097374 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:33.097393 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:33.097435 (+ 42us) server_negotiation.cc:407] Connection header received
0811 18:57:33.098227 (+ 792us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:33.098294 (+ 67us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:33.098681 (+ 387us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:33.099031 (+ 350us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:33.099894 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:33.101962 (+ 2068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:33.103114 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:33.103798 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:33.103943 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:33.106986 (+ 3043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:33.106993 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:33.107010 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:33.107042 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:33.108977 (+ 1935us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:33.109487 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:33.109493 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:33.109499 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:33.109622 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:33.109913 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:33.109916 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:33.109918 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:33.110468 (+ 550us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:33.110647 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:33.111384 (+ 737us) server_negotiation.cc:299] Negotiation successful
0811 18:57:33.111647 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":157,"threads_started":1}
I20260811 18:57:33.123198 19166 catalog_manager.cc:1347] Generated new cluster ID: bab8a6b15c4a416bad4f20f90f31738b
I20260811 18:57:33.123344 19166 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:57:33.134168 19166 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:57:33.135478 19166 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:57:33.148463 19166 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7c3b1394721845259008d27a9a7ff684: Generated new TSK 0
I20260811 18:57:33.149317 19166 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.180.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.180.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.180.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.129.keytab.
Entry for principal kudu/127.27.180.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.129.keytab.
I20260811 18:57:33.850605 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.27.180.148:55125/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.180.129:0
--local_ip_for_outbound_sockets=127.27.180.129
--webserver_interface=127.27.180.129
--webserver_port=0
--tserver_master_addrs=127.27.180.190:33551
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.129.keytab
--principal=kudu/127.27.180.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:44551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:57:34.014483 19226 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:57:34.015723 19226 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:57:34.015834 19226 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:57:34.022007 19226 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:57:34.022153 19226 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:57:34.022290 19226 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.180.129
I20260811 18:57:34.030262 19226 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:44551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.27.180.148:55125/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.129.keytab
--principal=kudu/127.27.180.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.180.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.180.129
--webserver_port=0
--tserver_master_addrs=127.27.180.190:33551
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19226
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.180.129
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:57:34.031922 19226 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:57:34.033355 19226 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:57:34.045996 19248 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:34.046459 19243 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:34.047189 19244 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:57:34.047538 19226 server_base.cc:1034] running on GCE node
I20260811 18:57:34.048130 19226 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:57:34.048895 19226 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:57:34.050071 19226 hybrid_clock.cc:648] HybridClock initialized: now 1786474654050042 us; error 43 us; skew 500 ppm
Aug 11 18:57:34 dist-test-slave-2j7r krb5kdc[18229](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474654, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:57:34.052852 19226 init.cc:377] Logged in from keytab as kudu/127.27.180.129@KRBTEST.COM (short username kudu)
I20260811 18:57:34.053982 19226 webserver.cc:460] Webserver started at http://127.27.180.129:46319/ using document root <none> and password file <none>
I20260811 18:57:34.054742 19226 fs_manager.cc:359] Metadata directory not provided
I20260811 18:57:34.054906 19226 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:57:34.055212 19226 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 18:57:34 dist-test-slave-2j7r krb5kdc[18229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474654, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.129@KRBTEST.COM for HTTP/127.27.180.148@KRBTEST.COM
I20260811 18:57:34.088881 19226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ts-0/data/instance:
uuid: "faa4fad679144835a25568d63b2cd2ee"
format_stamp: "Formatted at 2026-08-11 18:57:34 on dist-test-slave-2j7r"
server_key: "470954722595af40db37d4787a269eae"
server_key_iv: "55908cf7db5e547c18985041fe76fb36"
server_key_version: "kuduclusterkey@0"
I20260811 18:57:34.089931 19226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ts-0/wal/instance:
uuid: "faa4fad679144835a25568d63b2cd2ee"
format_stamp: "Formatted at 2026-08-11 18:57:34 on dist-test-slave-2j7r"
server_key: "470954722595af40db37d4787a269eae"
server_key_iv: "55908cf7db5e547c18985041fe76fb36"
server_key_version: "kuduclusterkey@0"
I20260811 18:57:34.095911 19226 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260811 18:57:34.125542 19258 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:34.138568 19226 fs_manager.cc:725] Time spent opening block manager: real 0.021s user 0.002s sys 0.000s
I20260811 18:57:34.138757 19226 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ts-0/wal
uuid: "faa4fad679144835a25568d63b2cd2ee"
format_stamp: "Formatted at 2026-08-11 18:57:34 on dist-test-slave-2j7r"
server_key: "470954722595af40db37d4787a269eae"
server_key_iv: "55908cf7db5e547c18985041fe76fb36"
server_key_version: "kuduclusterkey@0"
I20260811 18:57:34.138936 19226 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:57:34.168310 19226 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:57:34.172051 19226 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:57:34.172322 19226 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:57:34.173193 19226 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:57:34.174348 19226 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:57:34.174456 19226 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:34.174578 19226 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:57:34.174654 19226 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:34.185318 19226 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.129:35901
I20260811 18:57:34.186518 19226 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ts-0/data/info.pb
I20260811 18:57:34.191489 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 19226
I20260811 18:57:34.191618 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ts-0/wal/instance
I20260811 18:57:34.191983 28370 external_mini_cluster.cc:1428] Setting key 6d237e580fbf856af11dfe52500cb484
I20260811 18:57:34.193223 19473 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.129:35901 every 8 connection(s)
Aug 11 18:57:34 dist-test-slave-2j7r krb5kdc[18229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474654, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.129@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.27.180.130@KRBTEST.COM; defaulting to no policy
I20260811 18:57:34.213126 19484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:34.196364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.129:38413 (local address 127.27.180.190:33551)
0811 18:57:34.199852 (+ 3488us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:34.199856 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:34.199874 (+ 18us) server_negotiation.cc:407] Connection header received
0811 18:57:34.199920 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:34.199923 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:34.199975 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:34.200086 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:34.201718 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:34.202610 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:34.204486 (+ 1876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:34.204890 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:34.205004 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:34.208390 (+ 3386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:34.208394 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:34.208397 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:34.208427 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:34.210361 (+ 1934us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:34.211068 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:34.211072 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:34.211074 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:34.211175 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:34.211657 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:34.211662 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:34.211663 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:34.211932 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:34.212046 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:34.212743 (+ 697us) server_negotiation.cc:299] Negotiation successful
0811 18:57:34.212900 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3393,"thread_start_us":89,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.180.130@KRBTEST.COM" created.
I20260811 18:57:34.214746 19481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:34.195556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33551 (local address 127.27.180.129:38413)
0811 18:57:34.197185 (+ 1629us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:34.197243 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:34.198273 (+ 1030us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:34.200340 (+ 2067us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:34.200351 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:34.200741 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:57:34.201509 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:34.201535 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:34.202782 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:34.202785 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:34.204316 (+ 1531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:34.204330 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:34.205143 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:34.205149 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:34.205473 (+ 324us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:34.206275 (+ 802us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:57:34.206298 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:57:34.208144 (+ 1846us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:57:34.210500 (+ 2356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:34.210507 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:34.210525 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:34.210919 (+ 394us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:34.211289 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:34.211293 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:34.211295 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:34.211499 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:34.212058 (+ 559us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:57:34.212066 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:57:34.212422 (+ 356us) client_negotiation.cc:769] Sending connection context
0811 18:57:34.212635 (+ 213us) client_negotiation.cc:240] Negotiation successful
0811 18:57:34.212988 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1357,"thread_start_us":147,"threads_started":1}
I20260811 18:57:34.216506 19476 heartbeater.cc:347] Connected to a master server at 127.27.180.190:33551
I20260811 18:57:34.216753 19476 heartbeater.cc:464] Registering TS with master...
I20260811 18:57:34.217379 19476 heartbeater.cc:511] Master 127.27.180.190:33551 requested a full tablet report, sending...
I20260811 18:57:34.219290 19083 ts_manager.cc:194] Registered new tserver with Master: faa4fad679144835a25568d63b2cd2ee (127.27.180.129:35901)
I20260811 18:57:34.220875 19083 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.180.129@KRBTEST.COM'} at 127.27.180.129:38413
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.180.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.130.keytab.
Entry for principal kudu/127.27.180.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.130.keytab.
I20260811 18:57:34.236414 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.27.180.148:55125/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.180.130:0
--local_ip_for_outbound_sockets=127.27.180.130
--webserver_interface=127.27.180.130
--webserver_port=0
--tserver_master_addrs=127.27.180.190:33551
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.130.keytab
--principal=kudu/127.27.180.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:44551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 18:57:34 dist-test-slave-2j7r krb5kdc[18229](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786474654, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260811 18:57:34.385258 19488 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:57:34.385619 19488 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:57:34.385707 19488 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:57:34.391523 19488 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:57:34.391646 19488 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:57:34.391763 19488 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.180.130
I20260811 18:57:34.399391 19488 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:44551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.27.180.148:55125/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.130.keytab
--principal=kudu/127.27.180.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.180.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.180.130
--webserver_port=0
--tserver_master_addrs=127.27.180.190:33551
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19488
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.180.130
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:57:34.400921 19488 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:57:34.402310 19488 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:57:34.415115 19508 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:34.415534 19512 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:34.416368 19510 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:57:34.416707 19488 server_base.cc:1034] running on GCE node
I20260811 18:57:34.417327 19488 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:57:34.418155 19488 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:57:34.419354 19488 hybrid_clock.cc:648] HybridClock initialized: now 1786474654419318 us; error 52 us; skew 500 ppm
Aug 11 18:57:34 dist-test-slave-2j7r krb5kdc[18229](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474654, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:57:34.423173 19488 init.cc:377] Logged in from keytab as kudu/127.27.180.130@KRBTEST.COM (short username kudu)
I20260811 18:57:34.424397 19488 webserver.cc:460] Webserver started at http://127.27.180.130:35245/ using document root <none> and password file <none>
I20260811 18:57:34.425240 19488 fs_manager.cc:359] Metadata directory not provided
I20260811 18:57:34.425390 19488 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:57:34.425686 19488 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 18:57:34 dist-test-slave-2j7r krb5kdc[18229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474654, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.130@KRBTEST.COM for HTTP/127.27.180.148@KRBTEST.COM
I20260811 18:57:34.443557 19488 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ts-1/data/instance:
uuid: "aeb897b1c02c482ba7b4ec29b3d28484"
format_stamp: "Formatted at 2026-08-11 18:57:34 on dist-test-slave-2j7r"
server_key: "3eb0932bd0c837c168b88de26db62ea4"
server_key_iv: "f2a12220fc99ffdc7e6ab185ba6a57d5"
server_key_version: "kuduclusterkey@0"
I20260811 18:57:34.444532 19488 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ts-1/wal/instance:
uuid: "aeb897b1c02c482ba7b4ec29b3d28484"
format_stamp: "Formatted at 2026-08-11 18:57:34 on dist-test-slave-2j7r"
server_key: "3eb0932bd0c837c168b88de26db62ea4"
server_key_iv: "f2a12220fc99ffdc7e6ab185ba6a57d5"
server_key_version: "kuduclusterkey@0"
I20260811 18:57:34.453117 19488 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.004s sys 0.000s
I20260811 18:57:34.472549 19529 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:34.474464 19488 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260811 18:57:34.474633 19488 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ts-1/wal
uuid: "aeb897b1c02c482ba7b4ec29b3d28484"
format_stamp: "Formatted at 2026-08-11 18:57:34 on dist-test-slave-2j7r"
server_key: "3eb0932bd0c837c168b88de26db62ea4"
server_key_iv: "f2a12220fc99ffdc7e6ab185ba6a57d5"
server_key_version: "kuduclusterkey@0"
I20260811 18:57:34.474783 19488 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:57:34.498580 19488 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:57:34.502547 19488 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:57:34.502835 19488 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:57:34.503692 19488 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:57:34.505007 19488 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:57:34.505142 19488 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:34.505257 19488 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:57:34.505338 19488 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:34.517851 19488 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.130:41129
I20260811 18:57:34.518203 19729 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.130:41129 every 8 connection(s)
I20260811 18:57:34.519058 19488 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ts-1/data/info.pb
I20260811 18:57:34.520278 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 19488
I20260811 18:57:34.520422 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ts-1/wal/instance
I20260811 18:57:34.520736 28370 external_mini_cluster.cc:1428] Setting key 149ab901fae21deb4292a7c8479c048e
WARNING: no policy specified for kudu/127.27.180.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.180.131@KRBTEST.COM" created.
Aug 11 18:57:34 dist-test-slave-2j7r krb5kdc[18229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474654, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.130@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM
I20260811 18:57:34.539445 19484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:34.520938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.130:45913 (local address 127.27.180.190:33551)
0811 18:57:34.521074 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:34.521078 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:34.522143 (+ 1065us) server_negotiation.cc:407] Connection header received
0811 18:57:34.525855 (+ 3712us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:34.525859 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:34.525913 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:34.526010 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:34.527496 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:34.528288 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:34.529994 (+ 1706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:34.530373 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:34.530487 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:34.535407 (+ 4920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:34.535410 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:34.535413 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:34.535441 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:34.536901 (+ 1460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:34.537577 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:34.537581 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:34.537582 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:34.537678 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:34.538085 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:34.538089 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:34.538090 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:34.538332 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:34.538430 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:34.539089 (+ 659us) server_negotiation.cc:299] Negotiation successful
0811 18:57:34.539253 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 18:57:34.540449 19733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:34.521421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33551 (local address 127.27.180.130:45913)
0811 18:57:34.521984 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:34.522042 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:34.525591 (+ 3549us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:34.526220 (+ 629us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:34.526229 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:34.526602 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:57:34.527299 (+ 697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:34.527321 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:34.528477 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:34.528480 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:34.529854 (+ 1374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:34.529867 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:34.530550 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:34.530554 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:34.530751 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:34.531402 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:57:34.531423 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:57:34.535212 (+ 3789us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:57:34.537032 (+ 1820us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:34.537039 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:34.537058 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:34.537457 (+ 399us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:34.537773 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:34.537776 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:34.537778 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:34.537978 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:34.538440 (+ 462us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:57:34.538447 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:57:34.538788 (+ 341us) client_negotiation.cc:769] Sending connection context
0811 18:57:34.538999 (+ 211us) client_negotiation.cc:240] Negotiation successful
0811 18:57:34.539348 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":145,"threads_started":1}
I20260811 18:57:34.541899 19730 heartbeater.cc:347] Connected to a master server at 127.27.180.190:33551
I20260811 18:57:34.542209 19730 heartbeater.cc:464] Registering TS with master...
I20260811 18:57:34.542894 19730 heartbeater.cc:511] Master 127.27.180.190:33551 requested a full tablet report, sending...
I20260811 18:57:34.544212 19083 ts_manager.cc:194] Registered new tserver with Master: aeb897b1c02c482ba7b4ec29b3d28484 (127.27.180.130:41129)
I20260811 18:57:34.545048 19083 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.180.130@KRBTEST.COM'} at 127.27.180.130:45913
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.180.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.131.keytab.
Entry for principal kudu/127.27.180.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.131.keytab.
I20260811 18:57:34.562997 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.27.180.148:55125/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.180.131:0
--local_ip_for_outbound_sockets=127.27.180.131
--webserver_interface=127.27.180.131
--webserver_port=0
--tserver_master_addrs=127.27.180.190:33551
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.131.keytab
--principal=kudu/127.27.180.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:44551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 18:57:34 dist-test-slave-2j7r krb5kdc[18229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474654, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.190@KRBTEST.COM for HTTP/127.27.180.148@KRBTEST.COM
W20260811 18:57:34.725155 19738 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:57:34.725567 19738 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:57:34.725675 19738 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:57:34.731752 19738 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:57:34.731911 19738 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:57:34.732043 19738 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.180.131
I20260811 18:57:34.739929 19738 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:44551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.27.180.148:55125/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.131.keytab
--principal=kudu/127.27.180.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.180.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.180.131
--webserver_port=0
--tserver_master_addrs=127.27.180.190:33551
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19738
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.180.131
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:57:34.741659 19738 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:57:34.743077 19738 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:57:34.757423 19758 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:57:34.758764 19738 server_base.cc:1034] running on GCE node
W20260811 18:57:34.758814 19753 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:34.765596 19767 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:57:34.766268 19738 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:57:34.767104 19738 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:57:34.770409 19738 hybrid_clock.cc:648] HybridClock initialized: now 1786474654770320 us; error 101 us; skew 500 ppm
Aug 11 18:57:34 dist-test-slave-2j7r krb5kdc[18229](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474654, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:57:34.774580 19738 init.cc:377] Logged in from keytab as kudu/127.27.180.131@KRBTEST.COM (short username kudu)
I20260811 18:57:34.775799 19738 webserver.cc:460] Webserver started at http://127.27.180.131:38665/ using document root <none> and password file <none>
I20260811 18:57:34.776592 19738 fs_manager.cc:359] Metadata directory not provided
I20260811 18:57:34.776741 19738 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:57:34.777055 19738 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 18:57:34 dist-test-slave-2j7r krb5kdc[18229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474654, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.131@KRBTEST.COM for HTTP/127.27.180.148@KRBTEST.COM
I20260811 18:57:34.795504 19738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ts-2/data/instance:
uuid: "63d442048614417ab0e8215eb33a4e59"
format_stamp: "Formatted at 2026-08-11 18:57:34 on dist-test-slave-2j7r"
server_key: "ad04076b015ee509a7f5116344b537bc"
server_key_iv: "82dc49e9b0224dfac504d805ae264c76"
server_key_version: "kuduclusterkey@0"
I20260811 18:57:34.796298 19738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ts-2/wal/instance:
uuid: "63d442048614417ab0e8215eb33a4e59"
format_stamp: "Formatted at 2026-08-11 18:57:34 on dist-test-slave-2j7r"
server_key: "ad04076b015ee509a7f5116344b537bc"
server_key_iv: "82dc49e9b0224dfac504d805ae264c76"
server_key_version: "kuduclusterkey@0"
I20260811 18:57:34.821566 19738 fs_manager.cc:691] Time spent creating directory manager: real 0.025s user 0.001s sys 0.003s
I20260811 18:57:34.845105 19778 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:34.854471 19738 fs_manager.cc:725] Time spent opening block manager: real 0.018s user 0.004s sys 0.000s
I20260811 18:57:34.854657 19738 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ts-2/wal
uuid: "63d442048614417ab0e8215eb33a4e59"
format_stamp: "Formatted at 2026-08-11 18:57:34 on dist-test-slave-2j7r"
server_key: "ad04076b015ee509a7f5116344b537bc"
server_key_iv: "82dc49e9b0224dfac504d805ae264c76"
server_key_version: "kuduclusterkey@0"
I20260811 18:57:34.854813 19738 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:57:34.873261 19738 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:57:34.876796 19738 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:57:34.876981 19738 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:57:34.877691 19738 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:57:34.878682 19738 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:57:34.878751 19738 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:34.878861 19738 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:57:34.878893 19738 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:34.889652 19738 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.131:39189
I20260811 18:57:34.889690 20009 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.131:39189 every 8 connection(s)
I20260811 18:57:34.891054 19738 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ts-2/data/info.pb
I20260811 18:57:34.895968 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 19738
I20260811 18:57:34.896095 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ts-2/wal/instance
I20260811 18:57:34.896431 28370 external_mini_cluster.cc:1428] Setting key 872e2d412b74cf238ddf3b496e9f1d96
Aug 11 18:57:34 dist-test-slave-2j7r krb5kdc[18229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474654, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.131@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM
I20260811 18:57:34.907250 19484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:34.893159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.131:40277 (local address 127.27.180.190:33551)
0811 18:57:34.893324 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:34.893330 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:34.894379 (+ 1049us) server_negotiation.cc:407] Connection header received
0811 18:57:34.895643 (+ 1264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:34.895647 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:34.895710 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:34.895799 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:34.897351 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:34.898184 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:34.899815 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:34.900165 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:34.900298 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:34.903165 (+ 2867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:34.903169 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:34.903172 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:34.903201 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:34.904763 (+ 1562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:34.905416 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:34.905420 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:34.905422 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:34.905510 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:34.905878 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:34.905881 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:34.905883 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:34.906103 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:34.906198 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:34.906861 (+ 663us) server_negotiation.cc:299] Negotiation successful
0811 18:57:34.906989 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 18:57:34.908156 20020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:34.893643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33551 (local address 127.27.180.131:40277)
0811 18:57:34.894204 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:34.894269 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:34.895381 (+ 1112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:34.896111 (+ 730us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:34.896121 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:34.896483 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:57:34.897163 (+ 680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:34.897184 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:34.898358 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:34.898361 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:34.899671 (+ 1310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:34.899684 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:34.900349 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:34.900355 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:34.900558 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:34.901239 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:57:34.901261 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:57:34.902981 (+ 1720us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:57:34.904885 (+ 1904us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:34.904891 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:34.904907 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:34.905312 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:34.905599 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:34.905603 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:34.905604 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:34.905788 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:34.906266 (+ 478us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:57:34.906273 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:57:34.906570 (+ 297us) client_negotiation.cc:769] Sending connection context
0811 18:57:34.906763 (+ 193us) client_negotiation.cc:240] Negotiation successful
0811 18:57:34.907076 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":147,"threads_started":1}
I20260811 18:57:34.910029 20012 heartbeater.cc:347] Connected to a master server at 127.27.180.190:33551
I20260811 18:57:34.910290 20012 heartbeater.cc:464] Registering TS with master...
I20260811 18:57:34.910873 20012 heartbeater.cc:511] Master 127.27.180.190:33551 requested a full tablet report, sending...
I20260811 18:57:34.912174 19083 ts_manager.cc:194] Registered new tserver with Master: 63d442048614417ab0e8215eb33a4e59 (127.27.180.131:39189)
I20260811 18:57:34.912871 19083 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.180.131@KRBTEST.COM'} at 127.27.180.131:40277
I20260811 18:57:34.925205 28370 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 18:57:34.926146 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 19226
I20260811 18:57:34.954064 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 19488
I20260811 18:57:34.965173 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 19738
I20260811 18:57:34.978451 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 19032
I20260811 18:57:34.997509 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.190:33551
--webserver_interface=127.27.180.190
--webserver_port=36921
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab
--principal=kudu/127.27.180.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:44551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.180.190:33551
--ranger_config_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.27.180.148:55125/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.27.180.148:55125/kms with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:57:35.108108 20021 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:57:35.108358 20021 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:57:35.108394 20021 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:57:35.112274 20021 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:57:35.112324 20021 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:57:35.112342 20021 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:57:35.112357 20021 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:57:35.112371 20021 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:57:35.117702 20021 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:44551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.27.180.148:55125/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.27.180.190:33551
--ranger_config_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab
--principal=kudu/127.27.180.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.190:33551
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.180.190
--webserver_port=36921
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20021
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:57:35.118630 20021 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:57:35.119558 20021 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:57:35.128006 20037 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:35.128093 20035 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:35.128044 20041 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:57:35.128170 20021 server_base.cc:1034] running on GCE node
I20260811 18:57:35.128758 20021 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:57:35.129828 20021 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:57:35.130982 20021 hybrid_clock.cc:648] HybridClock initialized: now 1786474655130968 us; error 26 us; skew 500 ppm
Aug 11 18:57:35 dist-test-slave-2j7r krb5kdc[18229](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474655, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:57:35.133579 20021 init.cc:377] Logged in from keytab as kudu/127.27.180.190@KRBTEST.COM (short username kudu)
I20260811 18:57:35.134434 20021 webserver.cc:460] Webserver started at http://127.27.180.190:36921/ using document root <none> and password file <none>
I20260811 18:57:35.134928 20021 fs_manager.cc:359] Metadata directory not provided
I20260811 18:57:35.134997 20021 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 18:57:35 dist-test-slave-2j7r krb5kdc[18229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474655, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.190@KRBTEST.COM for HTTP/127.27.180.148@KRBTEST.COM
I20260811 18:57:35.218808 20021 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260811 18:57:35.221235 20059 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:35.222652 20021 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:57:35.222771 20021 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/master-0/wal
uuid: "7c3b1394721845259008d27a9a7ff684"
format_stamp: "Formatted at 2026-08-11 18:57:33 on dist-test-slave-2j7r"
server_key: "9ab7aaf58fc40c1cfb6df7c0ed60cd32"
server_key_iv: "a2d3d571dfa9f746b809b3e19386c270"
server_key_version: "kuduclusterkey@0"
I20260811 18:57:35.223352 20021 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:57:35.237010 20021 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:57:35.239660 20021 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:57:35.239836 20021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:57:35.246980 20021 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.190:33551
I20260811 18:57:35.247001 20163 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.190:33551 every 8 connection(s)
I20260811 18:57:35.248046 20021 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
I20260811 18:57:35.252332 20164 sys_catalog.cc:263] Verifying existing consensus state
I20260811 18:57:35.253995 20164 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c3b1394721845259008d27a9a7ff684: Bootstrap starting.
I20260811 18:57:35.255995 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 20021
I20260811 18:57:35.256532 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.27.180.148:55125/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.180.129:35901
--local_ip_for_outbound_sockets=127.27.180.129
--tserver_master_addrs=127.27.180.190:33551
--webserver_port=46319
--webserver_interface=127.27.180.129
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.129.keytab
--principal=kudu/127.27.180.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:44551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.27.180.148:55125/kms with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 18:57:35.271034 20164 log.cc:824] T 00000000000000000000000000000000 P 7c3b1394721845259008d27a9a7ff684: Log is configured to *not* fsync() on all Append() calls
I20260811 18:57:35.277822 20164 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c3b1394721845259008d27a9a7ff684: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 18:57:35.278195 20164 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c3b1394721845259008d27a9a7ff684: Bootstrap complete.
I20260811 18:57:35.282336 20164 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7c3b1394721845259008d27a9a7ff684 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c3b1394721845259008d27a9a7ff684" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 33551 } }
I20260811 18:57:35.282709 20164 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7c3b1394721845259008d27a9a7ff684 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c3b1394721845259008d27a9a7ff684, State: Initialized, Role: FOLLOWER
I20260811 18:57:35.283210 20164 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7c3b1394721845259008d27a9a7ff684 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c3b1394721845259008d27a9a7ff684" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 33551 } }
I20260811 18:57:35.283329 20164 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7c3b1394721845259008d27a9a7ff684 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:57:35.283393 20164 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7c3b1394721845259008d27a9a7ff684 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:57:35.283483 20164 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7c3b1394721845259008d27a9a7ff684 [term 1 FOLLOWER]: Advancing to term 2
I20260811 18:57:35.285347 20164 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7c3b1394721845259008d27a9a7ff684 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c3b1394721845259008d27a9a7ff684" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 33551 } }
I20260811 18:57:35.285655 20164 leader_election.cc:302] T 00000000000000000000000000000000 P 7c3b1394721845259008d27a9a7ff684 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7c3b1394721845259008d27a9a7ff684; no voters:
I20260811 18:57:35.285956 20164 leader_election.cc:288] T 00000000000000000000000000000000 P 7c3b1394721845259008d27a9a7ff684 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 18:57:35.286090 20171 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7c3b1394721845259008d27a9a7ff684 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 18:57:35.286337 20171 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7c3b1394721845259008d27a9a7ff684 [term 2 LEADER]: Becoming Leader. State: Replica: 7c3b1394721845259008d27a9a7ff684, State: Running, Role: LEADER
I20260811 18:57:35.286652 20171 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7c3b1394721845259008d27a9a7ff684 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c3b1394721845259008d27a9a7ff684" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 33551 } }
I20260811 18:57:35.287124 20164 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7c3b1394721845259008d27a9a7ff684 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:57:35.288195 20173 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c3b1394721845259008d27a9a7ff684 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "7c3b1394721845259008d27a9a7ff684" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c3b1394721845259008d27a9a7ff684" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 33551 } } }
I20260811 18:57:35.288324 20173 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c3b1394721845259008d27a9a7ff684 [sys.catalog]: This master's current role is: LEADER
I20260811 18:57:35.288560 20174 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c3b1394721845259008d27a9a7ff684 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7c3b1394721845259008d27a9a7ff684. Latest consensus state: current_term: 2 leader_uuid: "7c3b1394721845259008d27a9a7ff684" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c3b1394721845259008d27a9a7ff684" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 33551 } } }
I20260811 18:57:35.288636 20174 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c3b1394721845259008d27a9a7ff684 [sys.catalog]: This master's current role is: LEADER
I20260811 18:57:35.289198 20191 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:57:35.289601 20164 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 18:57:35.295538 20191 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:57:35.296453 20191 catalog_manager.cc:1259] Loaded cluster ID: bab8a6b15c4a416bad4f20f90f31738b
I20260811 18:57:35.296516 20191 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:57:35.300016 20191 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:57:35.300848 20191 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 7c3b1394721845259008d27a9a7ff684: Loaded TSK: 0
I20260811 18:57:35.302160 20191 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 18:57:35.392328 20167 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:57:35.392551 20167 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:57:35.392611 20167 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:57:35.396636 20167 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:57:35.396693 20167 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:57:35.396770 20167 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.180.129
I20260811 18:57:35.403855 20167 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:44551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.27.180.148:55125/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.129.keytab
--principal=kudu/127.27.180.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.180.129:35901
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.180.129
--webserver_port=46319
--tserver_master_addrs=127.27.180.190:33551
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20167
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.180.129
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:57:35.405109 20167 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:57:35.406292 20167 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:57:35.416013 20234 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:35.416013 20236 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:57:35.416411 20167 server_base.cc:1034] running on GCE node
W20260811 18:57:35.416625 20229 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:57:35.417026 20167 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:57:35.417650 20167 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:57:35.419869 20167 hybrid_clock.cc:648] HybridClock initialized: now 1786474655419846 us; error 33 us; skew 500 ppm
Aug 11 18:57:35 dist-test-slave-2j7r krb5kdc[18229](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474655, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:57:35.422767 20167 init.cc:377] Logged in from keytab as kudu/127.27.180.129@KRBTEST.COM (short username kudu)
I20260811 18:57:35.423671 20167 webserver.cc:460] Webserver started at http://127.27.180.129:46319/ using document root <none> and password file <none>
I20260811 18:57:35.424197 20167 fs_manager.cc:359] Metadata directory not provided
I20260811 18:57:35.424289 20167 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 18:57:35 dist-test-slave-2j7r krb5kdc[18229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474655, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.129@KRBTEST.COM for HTTP/127.27.180.148@KRBTEST.COM
I20260811 18:57:35.448975 20167 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 18:57:35.453608 20251 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:35.455194 20167 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260811 18:57:35.455340 20167 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ts-0/wal
uuid: "faa4fad679144835a25568d63b2cd2ee"
format_stamp: "Formatted at 2026-08-11 18:57:34 on dist-test-slave-2j7r"
server_key: "470954722595af40db37d4787a269eae"
server_key_iv: "55908cf7db5e547c18985041fe76fb36"
server_key_version: "kuduclusterkey@0"
I20260811 18:57:35.456051 20167 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:57:35.475392 20167 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:57:35.478353 20167 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:57:35.478549 20167 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:57:35.479230 20167 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:57:35.480165 20167 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:57:35.480221 20167 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:35.480269 20167 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:57:35.480298 20167 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:35.491322 20167 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.129:35901
I20260811 18:57:35.491406 20464 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.129:35901 every 8 connection(s)
I20260811 18:57:35.493800 20167 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ts-0/data/info.pb
I20260811 18:57:35.496532 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 20167
I20260811 18:57:35.497037 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.27.180.148:55125/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.180.130:41129
--local_ip_for_outbound_sockets=127.27.180.130
--tserver_master_addrs=127.27.180.190:33551
--webserver_port=35245
--webserver_interface=127.27.180.130
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.130.keytab
--principal=kudu/127.27.180.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:44551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.27.180.148:55125/kms with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 18:57:35 dist-test-slave-2j7r krb5kdc[18229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474655, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.129@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM
I20260811 18:57:35.524979 20472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:35.496493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.129:43063 (local address 127.27.180.190:33551)
0811 18:57:35.498224 (+ 1731us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:35.498248 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:35.498663 (+ 415us) server_negotiation.cc:407] Connection header received
0811 18:57:35.500244 (+ 1581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:35.500262 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:35.500706 (+ 444us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:35.501062 (+ 356us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:35.502633 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:35.504200 (+ 1567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:35.506001 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:35.506712 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:35.506878 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:35.518256 (+ 11378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:35.518265 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:35.518286 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:35.518327 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:35.520499 (+ 2172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:35.521389 (+ 890us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:35.521396 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:35.521402 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:35.521564 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:35.522015 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:35.522018 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:35.522020 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:35.522568 (+ 548us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:35.522711 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:35.523491 (+ 780us) server_negotiation.cc:299] Negotiation successful
0811 18:57:35.523830 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1542,"thread_start_us":128,"threads_started":1}
I20260811 18:57:35.526607 20474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:35.496554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33551 (local address 127.27.180.129:43063)
0811 18:57:35.498489 (+ 1935us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:35.498532 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:35.499694 (+ 1162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:35.501283 (+ 1589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:35.501292 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:35.501648 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:57:35.502411 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:35.502436 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:35.504386 (+ 1950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:35.504390 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:35.505829 (+ 1439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:35.505842 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:35.507117 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:35.507121 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:35.507323 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:35.509279 (+ 1956us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:57:35.509304 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:57:35.517669 (+ 8365us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:57:35.520678 (+ 3009us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:35.520685 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:35.520701 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:35.521228 (+ 527us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:35.521693 (+ 465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:35.521696 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:35.521698 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:35.521892 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:35.522775 (+ 883us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:57:35.522782 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:57:35.523111 (+ 329us) client_negotiation.cc:769] Sending connection context
0811 18:57:35.525381 (+ 2270us) client_negotiation.cc:240] Negotiation successful
0811 18:57:35.525712 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1759,"thread_start_us":140,"threads_started":1}
I20260811 18:57:35.528947 20465 heartbeater.cc:347] Connected to a master server at 127.27.180.190:33551
I20260811 18:57:35.529222 20465 heartbeater.cc:464] Registering TS with master...
I20260811 18:57:35.529825 20465 heartbeater.cc:511] Master 127.27.180.190:33551 requested a full tablet report, sending...
W20260811 18:57:35.532187 20465 heartbeater.cc:644] Failed to heartbeat to 127.27.180.190:33551 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 18:57:35.532332 20465 heartbeater.cc:464] Registering TS with master...
I20260811 18:57:35.532536 20465 heartbeater.cc:511] Master 127.27.180.190:33551 requested a full tablet report, sending...
I20260811 18:57:35.533520 20465 heartbeater.cc:464] Registering TS with master...
I20260811 18:57:35.533717 20465 heartbeater.cc:511] Master 127.27.180.190:33551 requested a full tablet report, sending...
W20260811 18:57:35.534703 20465 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 18:57:35.641809 20475 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:57:35.642081 20475 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:57:35.642141 20475 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:57:35.646286 20475 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:57:35.646339 20475 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:57:35.646409 20475 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.180.130
I20260811 18:57:35.652220 20475 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:44551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.27.180.148:55125/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.130.keytab
--principal=kudu/127.27.180.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.180.130:41129
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.180.130
--webserver_port=35245
--tserver_master_addrs=127.27.180.190:33551
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20475
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.180.130
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:57:35.653467 20475 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:57:35.654505 20475 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:57:35.666235 20491 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:35.666600 20488 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:35.666272 20483 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:57:35.666827 20475 server_base.cc:1034] running on GCE node
I20260811 18:57:35.667263 20475 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:57:35.667939 20475 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:57:35.669164 20475 hybrid_clock.cc:648] HybridClock initialized: now 1786474655669134 us; error 43 us; skew 500 ppm
Aug 11 18:57:35 dist-test-slave-2j7r krb5kdc[18229](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474655, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:57:35.672659 20475 init.cc:377] Logged in from keytab as kudu/127.27.180.130@KRBTEST.COM (short username kudu)
I20260811 18:57:35.673800 20475 webserver.cc:460] Webserver started at http://127.27.180.130:35245/ using document root <none> and password file <none>
I20260811 18:57:35.674543 20475 fs_manager.cc:359] Metadata directory not provided
I20260811 18:57:35.674661 20475 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 18:57:35 dist-test-slave-2j7r krb5kdc[18229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474655, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.130@KRBTEST.COM for HTTP/127.27.180.148@KRBTEST.COM
I20260811 18:57:35.698132 20475 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260811 18:57:35.701581 20506 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:35.703351 20475 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260811 18:57:35.703531 20475 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ts-1/wal
uuid: "aeb897b1c02c482ba7b4ec29b3d28484"
format_stamp: "Formatted at 2026-08-11 18:57:34 on dist-test-slave-2j7r"
server_key: "3eb0932bd0c837c168b88de26db62ea4"
server_key_iv: "f2a12220fc99ffdc7e6ab185ba6a57d5"
server_key_version: "kuduclusterkey@0"
I20260811 18:57:35.704355 20475 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:57:35.728277 20475 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:57:35.734505 20475 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:57:35.734750 20475 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:57:35.735564 20475 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:57:35.736708 20475 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:57:35.736774 20475 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:35.736829 20475 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:57:35.736857 20475 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:35.747594 20475 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.130:41129
I20260811 18:57:35.748973 20475 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ts-1/data/info.pb
I20260811 18:57:35.753237 20729 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.130:41129 every 8 connection(s)
I20260811 18:57:35.755149 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 20475
I20260811 18:57:35.755630 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.27.180.148:55125/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.180.131:39189
--local_ip_for_outbound_sockets=127.27.180.131
--tserver_master_addrs=127.27.180.190:33551
--webserver_port=38665
--webserver_interface=127.27.180.131
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.131.keytab
--principal=kudu/127.27.180.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:44551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.27.180.148:55125/kms with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 18:57:35 dist-test-slave-2j7r krb5kdc[18229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474655, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.130@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM
I20260811 18:57:35.772270 20472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:35.756359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.130:40341 (local address 127.27.180.190:33551)
0811 18:57:35.756493 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:35.756498 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:35.758144 (+ 1646us) server_negotiation.cc:407] Connection header received
0811 18:57:35.759403 (+ 1259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:35.759407 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:35.759467 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:35.759523 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:35.761325 (+ 1802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:35.762128 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:35.763605 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:35.763905 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:35.763999 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:35.766866 (+ 2867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:35.766871 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:35.766874 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:35.766904 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:35.768530 (+ 1626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:35.769205 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:35.769209 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:35.769211 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:35.769308 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:35.769753 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:35.769757 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:35.769759 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:35.769991 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:35.770080 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:35.771949 (+ 1869us) server_negotiation.cc:299] Negotiation successful
0811 18:57:35.772095 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 18:57:35.774253 20732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:35.755238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33551 (local address 127.27.180.130:40341)
0811 18:57:35.757939 (+ 2701us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:35.757997 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:35.759125 (+ 1128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:35.759970 (+ 845us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:35.759979 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:35.760408 (+ 429us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:57:35.761127 (+ 719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:35.761151 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:35.762300 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:35.762304 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:35.763472 (+ 1168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:35.763483 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:35.764168 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:35.764172 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:35.764380 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:35.765000 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:57:35.765021 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:57:35.766685 (+ 1664us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:57:35.768671 (+ 1986us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:35.768678 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:35.768707 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:35.769057 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:35.769426 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:35.769429 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:35.769431 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:35.769617 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:35.770154 (+ 537us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:57:35.770160 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:57:35.770486 (+ 326us) client_negotiation.cc:769] Sending connection context
0811 18:57:35.772757 (+ 2271us) client_negotiation.cc:240] Negotiation successful
0811 18:57:35.773140 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2462,"thread_start_us":146,"threads_started":1}
I20260811 18:57:35.775924 20730 heartbeater.cc:347] Connected to a master server at 127.27.180.190:33551
I20260811 18:57:35.776155 20730 heartbeater.cc:464] Registering TS with master...
I20260811 18:57:35.776721 20730 heartbeater.cc:511] Master 127.27.180.190:33551 requested a full tablet report, sending...
W20260811 18:57:35.778491 20730 heartbeater.cc:644] Failed to heartbeat to 127.27.180.190:33551 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 18:57:35.778607 20730 heartbeater.cc:464] Registering TS with master...
I20260811 18:57:35.778776 20730 heartbeater.cc:511] Master 127.27.180.190:33551 requested a full tablet report, sending...
I20260811 18:57:35.779568 20730 heartbeater.cc:464] Registering TS with master...
I20260811 18:57:35.779736 20730 heartbeater.cc:511] Master 127.27.180.190:33551 requested a full tablet report, sending...
W20260811 18:57:35.780462 20730 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 18:57:35.901401 20733 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:57:35.901651 20733 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:57:35.901711 20733 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:57:35.906116 20733 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:57:35.906201 20733 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:57:35.906286 20733 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.180.131
I20260811 18:57:35.913242 20733 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:44551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.27.180.148:55125/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.131.keytab
--principal=kudu/127.27.180.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.180.131:39189
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.180.131
--webserver_port=38665
--tserver_master_addrs=127.27.180.190:33551
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20733
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.180.131
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:57:35.914469 20733 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:57:35.915759 20733 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:57:35.925798 20748 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:35.925854 20746 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:35.925954 20752 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:57:35.926303 20733 server_base.cc:1034] running on GCE node
I20260811 18:57:35.926785 20733 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:57:35.927600 20733 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:57:35.928815 20733 hybrid_clock.cc:648] HybridClock initialized: now 1786474655928788 us; error 30 us; skew 500 ppm
Aug 11 18:57:35 dist-test-slave-2j7r krb5kdc[18229](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474655, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:57:35.932315 20733 init.cc:377] Logged in from keytab as kudu/127.27.180.131@KRBTEST.COM (short username kudu)
I20260811 18:57:35.933421 20733 webserver.cc:460] Webserver started at http://127.27.180.131:38665/ using document root <none> and password file <none>
I20260811 18:57:35.934123 20733 fs_manager.cc:359] Metadata directory not provided
I20260811 18:57:35.934248 20733 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 18:57:35 dist-test-slave-2j7r krb5kdc[18229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474655, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.131@KRBTEST.COM for HTTP/127.27.180.148@KRBTEST.COM
I20260811 18:57:35.952499 20733 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 18:57:35.965615 20764 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:35.967540 20733 fs_manager.cc:725] Time spent opening block manager: real 0.014s user 0.002s sys 0.000s
I20260811 18:57:35.967710 20733 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ts-2/wal
uuid: "63d442048614417ab0e8215eb33a4e59"
format_stamp: "Formatted at 2026-08-11 18:57:34 on dist-test-slave-2j7r"
server_key: "ad04076b015ee509a7f5116344b537bc"
server_key_iv: "82dc49e9b0224dfac504d805ae264c76"
server_key_version: "kuduclusterkey@0"
I20260811 18:57:35.968493 20733 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:57:35.992540 20733 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:57:35.995538 20733 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:57:35.995713 20733 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:57:35.996335 20733 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:57:35.997275 20733 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:57:35.997337 20733 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:35.997399 20733 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:57:35.997429 20733 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:36.008922 20733 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.131:39189
I20260811 18:57:36.010360 20733 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ts-2/data/info.pb
I20260811 18:57:36.011453 20974 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.131:39189 every 8 connection(s)
I20260811 18:57:36.019296 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 20733
Aug 11 18:57:36 dist-test-slave-2j7r krb5kdc[18229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474655, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.131@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM
I20260811 18:57:36.026386 20978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:36.012811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33551 (local address 127.27.180.131:58305)
0811 18:57:36.013349 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:36.013401 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:36.014254 (+ 853us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:36.014799 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:36.014806 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:36.015066 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:57:36.015560 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:36.015579 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:36.016719 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:36.016722 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:36.017659 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:36.017670 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:36.018298 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:36.018300 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:36.018443 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:36.019010 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:57:36.019034 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:57:36.020787 (+ 1753us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:57:36.023025 (+ 2238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:36.023031 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:36.023046 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:36.023385 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:36.023709 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:36.023712 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:36.023715 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:36.023910 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:36.024413 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:57:36.024419 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:57:36.024737 (+ 318us) client_negotiation.cc:769] Sending connection context
0811 18:57:36.024965 (+ 228us) client_negotiation.cc:240] Negotiation successful
0811 18:57:36.025338 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":135,"threads_started":1}
I20260811 18:57:36.027278 20472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:36.012298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.131:58305 (local address 127.27.180.190:33551)
0811 18:57:36.012479 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:36.012484 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:36.013520 (+ 1036us) server_negotiation.cc:407] Connection header received
0811 18:57:36.014473 (+ 953us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:36.014477 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:36.014536 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:36.014646 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:36.015734 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:36.016562 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:36.017800 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:36.018143 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:36.018255 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:36.020989 (+ 2734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:36.020993 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:36.020996 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:36.021027 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:36.022888 (+ 1861us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:36.023510 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:36.023513 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:36.023515 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:36.023604 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:36.024022 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:36.024026 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:36.024028 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:36.024258 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:36.024356 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:36.026777 (+ 2421us) server_negotiation.cc:299] Negotiation successful
0811 18:57:36.026908 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 18:57:36.028816 20976 heartbeater.cc:347] Connected to a master server at 127.27.180.190:33551
I20260811 18:57:36.029065 20976 heartbeater.cc:464] Registering TS with master...
I20260811 18:57:36.029659 20976 heartbeater.cc:511] Master 127.27.180.190:33551 requested a full tablet report, sending...
W20260811 18:57:36.031325 20976 heartbeater.cc:644] Failed to heartbeat to 127.27.180.190:33551 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 18:57:36.031445 20976 heartbeater.cc:464] Registering TS with master...
I20260811 18:57:36.031605 20976 heartbeater.cc:511] Master 127.27.180.190:33551 requested a full tablet report, sending...
I20260811 18:57:36.032428 20976 heartbeater.cc:464] Registering TS with master...
I20260811 18:57:36.032598 20976 heartbeater.cc:511] Master 127.27.180.190:33551 requested a full tablet report, sending...
W20260811 18:57:36.033427 20976 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 18:57:36.033885 20985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:36.020311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49212 (local address 127.27.180.190:33551)
0811 18:57:36.021467 (+ 1156us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:36.021471 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:36.021484 (+ 13us) server_negotiation.cc:407] Connection header received
0811 18:57:36.021517 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:36.021519 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:36.021564 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:36.021648 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:36.022679 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:36.023439 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:36.024668 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:36.025032 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:36.025171 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:36.026979 (+ 1808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:36.026983 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:36.026986 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:36.027011 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:36.030488 (+ 3477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:36.030988 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:36.030992 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:36.030994 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:36.031843 (+ 849us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:36.032159 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:36.032162 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:36.032164 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:36.032841 (+ 677us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:36.033047 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:36.033599 (+ 552us) server_negotiation.cc:299] Negotiation successful
0811 18:57:36.033731 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1082,"thread_start_us":97,"threads_started":1}
Aug 11 18:57:36 dist-test-slave-2j7r krb5kdc[18229](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786474656, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:57:36.534929 20465 heartbeater.cc:464] Registering TS with master...
I20260811 18:57:36.535210 20465 heartbeater.cc:511] Master 127.27.180.190:33551 requested a full tablet report, sending...
I20260811 18:57:36.536250 20095 ts_manager.cc:194] Registered new tserver with Master: faa4fad679144835a25568d63b2cd2ee (127.27.180.129:35901)
I20260811 18:57:36.537515 20095 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.180.129@KRBTEST.COM'} at 127.27.180.129:43063
I20260811 18:57:36.780689 20730 heartbeater.cc:464] Registering TS with master...
I20260811 18:57:36.781031 20730 heartbeater.cc:511] Master 127.27.180.190:33551 requested a full tablet report, sending...
I20260811 18:57:36.781891 20095 ts_manager.cc:194] Registered new tserver with Master: aeb897b1c02c482ba7b4ec29b3d28484 (127.27.180.130:41129)
I20260811 18:57:36.782696 20095 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.180.130@KRBTEST.COM'} at 127.27.180.130:40341
Aug 11 18:57:36 dist-test-slave-2j7r krb5kdc[18229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474656, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.190@KRBTEST.COM for HTTP/127.27.180.148@KRBTEST.COM
I20260811 18:57:37.033669 20976 heartbeater.cc:464] Registering TS with master...
I20260811 18:57:37.034008 20976 heartbeater.cc:511] Master 127.27.180.190:33551 requested a full tablet report, sending...
I20260811 18:57:37.034791 20095 ts_manager.cc:194] Registered new tserver with Master: 63d442048614417ab0e8215eb33a4e59 (127.27.180.131:39189)
I20260811 18:57:37.035446 20095 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.180.131@KRBTEST.COM'} at 127.27.180.131:58305
I20260811 18:57:37.046878 28370 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 18:57:37.058431 21012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:37.048418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47344 (local address 127.27.180.190:33551)
0811 18:57:37.048672 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:37.048675 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:37.048904 (+ 229us) server_negotiation.cc:407] Connection header received
0811 18:57:37.049020 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:37.049024 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:37.049073 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:37.049171 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:37.050960 (+ 1789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:37.051778 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:37.052623 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:37.052909 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:37.053015 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:37.054613 (+ 1598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:37.054617 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:37.054620 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:37.054650 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:37.056381 (+ 1731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:37.056870 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:37.056874 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:37.056876 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:37.056979 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:37.057347 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:37.057351 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:37.057453 (+ 102us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:37.057678 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:37.057834 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:37.058105 (+ 271us) server_negotiation.cc:299] Negotiation successful
0811 18:57:37.058236 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":92,"threads_started":1}
I20260811 18:57:37.062701 20095 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47344:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:37.067576 20095 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:37.094349 21022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:37.085297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.131:39189 (local address 127.0.0.1:51456)
0811 18:57:37.088076 (+ 2779us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:37.088125 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:37.088276 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:37.089175 (+ 899us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:37.089181 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:37.089203 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:57:37.089350 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:37.089358 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:37.090812 (+ 1454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:37.090815 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:37.092203 (+ 1388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:37.092217 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:37.093569 (+ 1352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:37.093573 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:37.093689 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:37.093731 (+ 42us) client_negotiation.cc:769] Sending connection context
0811 18:57:37.094033 (+ 302us) client_negotiation.cc:240] Negotiation successful
0811 18:57:37.094168 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2612,"thread_start_us":108,"threads_started":1}
I20260811 18:57:37.095062 21021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:37.086196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51456 (local address 127.27.180.131:39189)
0811 18:57:37.087836 (+ 1640us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:37.087842 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:37.088434 (+ 592us) server_negotiation.cc:407] Connection header received
0811 18:57:37.088604 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:37.088610 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:37.088823 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:37.088966 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:57:37.089481 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:37.090670 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:37.092372 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:37.093234 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:37.093359 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:37.093502 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:37.094822 (+ 1320us) server_negotiation.cc:299] Negotiation successful
0811 18:57:37.094929 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1536,"thread_start_us":118,"threads_started":1}
I20260811 18:57:37.098533 20860 tablet_service.cc:1467] Processing CreateTablet for tablet b016bfa0fb474da1a6fc334def67deba (DEFAULT_TABLE table=test-table [id=702f0c03f0424078bdd479c4aab85da5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.099757 20860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b016bfa0fb474da1a6fc334def67deba. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 18:57:37 dist-test-slave-2j7r krb5kdc[18229](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474656, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.190@KRBTEST.COM for HTTP/127.27.180.148@KRBTEST.COM
I20260811 18:57:37.109907 21029 tablet_bootstrap.cc:492] T b016bfa0fb474da1a6fc334def67deba P 63d442048614417ab0e8215eb33a4e59: Bootstrap starting.
I20260811 18:57:37.112530 21029 tablet_bootstrap.cc:654] T b016bfa0fb474da1a6fc334def67deba P 63d442048614417ab0e8215eb33a4e59: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:37.113497 21029 log.cc:824] T b016bfa0fb474da1a6fc334def67deba P 63d442048614417ab0e8215eb33a4e59: Log is configured to *not* fsync() on all Append() calls
I20260811 18:57:37.117406 21023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:37.086825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39120 (local address 127.27.180.130:41129)
0811 18:57:37.087457 (+ 632us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:37.087464 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:37.088612 (+ 1148us) server_negotiation.cc:407] Connection header received
0811 18:57:37.089004 (+ 392us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:37.089010 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:37.089215 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:37.097852 (+ 8637us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:57:37.110339 (+ 12487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:37.111616 (+ 1277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:37.115680 (+ 4064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:37.116481 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:37.116582 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:37.116714 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:37.117123 (+ 409us) server_negotiation.cc:299] Negotiation successful
0811 18:57:37.117241 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":518,"thread_start_us":126,"threads_started":1}
I20260811 18:57:37.117653 21029 tablet_bootstrap.cc:492] T b016bfa0fb474da1a6fc334def67deba P 63d442048614417ab0e8215eb33a4e59: No bootstrap required, opened a new log
I20260811 18:57:37.117846 21029 ts_tablet_manager.cc:1397] T b016bfa0fb474da1a6fc334def67deba P 63d442048614417ab0e8215eb33a4e59: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260811 18:57:37.119048 21026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:37.086637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.130:41129 (local address 127.0.0.1:39120)
0811 18:57:37.088526 (+ 1889us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:37.088539 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:37.088859 (+ 320us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:37.110027 (+ 21168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:37.110029 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:37.110051 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:57:37.110193 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:37.110203 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:37.111804 (+ 1601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:37.111807 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:37.112896 (+ 1089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:37.112913 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:37.116829 (+ 3916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:37.116832 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:37.116923 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:37.116947 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 18:57:37.118829 (+ 1882us) client_negotiation.cc:240] Negotiation successful
0811 18:57:37.118909 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1829,"thread_start_us":89,"threads_started":1}
I20260811 18:57:37.122049 21029 raft_consensus.cc:348] T b016bfa0fb474da1a6fc334def67deba P 63d442048614417ab0e8215eb33a4e59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aeb897b1c02c482ba7b4ec29b3d28484" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 41129 } } peers { permanent_uuid: "63d442048614417ab0e8215eb33a4e59" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 39189 } } peers { permanent_uuid: "faa4fad679144835a25568d63b2cd2ee" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 35901 } }
I20260811 18:57:37.123992 21029 raft_consensus.cc:374] T b016bfa0fb474da1a6fc334def67deba P 63d442048614417ab0e8215eb33a4e59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:37.124101 21029 raft_consensus.cc:729] T b016bfa0fb474da1a6fc334def67deba P 63d442048614417ab0e8215eb33a4e59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63d442048614417ab0e8215eb33a4e59, State: Initialized, Role: FOLLOWER
I20260811 18:57:37.125676 21020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:37.087691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.129:35901 (local address 127.0.0.1:33872)
0811 18:57:37.093937 (+ 6246us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:37.093951 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:37.094047 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:37.113064 (+ 19017us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:37.113067 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:37.113108 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:57:37.113194 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:37.113200 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:37.117660 (+ 4460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:37.117664 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:37.118726 (+ 1062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:37.118737 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:37.125260 (+ 6523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:37.125264 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:37.125371 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:37.125400 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 18:57:37.125455 (+ 55us) client_negotiation.cc:240] Negotiation successful
0811 18:57:37.125529 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6176,"thread_start_us":100,"threads_started":1}
I20260811 18:57:37.126072 21029 consensus_queue.cc:260] T b016bfa0fb474da1a6fc334def67deba P 63d442048614417ab0e8215eb33a4e59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aeb897b1c02c482ba7b4ec29b3d28484" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 41129 } } peers { permanent_uuid: "63d442048614417ab0e8215eb33a4e59" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 39189 } } peers { permanent_uuid: "faa4fad679144835a25568d63b2cd2ee" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 35901 } }
I20260811 18:57:37.126470 21027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:37.088105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33872 (local address 127.27.180.129:35901)
0811 18:57:37.088795 (+ 690us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:37.088801 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:37.109517 (+ 20716us) server_negotiation.cc:407] Connection header received
0811 18:57:37.109614 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:37.109620 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:37.109839 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:37.113355 (+ 3516us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:57:37.113512 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:37.115529 (+ 2017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:37.119950 (+ 4421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:37.120771 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:37.126029 (+ 5258us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:37.126178 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:37.126234 (+ 56us) server_negotiation.cc:299] Negotiation successful
0811 18:57:37.126334 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":585,"thread_start_us":94,"threads_started":1}
I20260811 18:57:37.127490 21029 ts_tablet_manager.cc:1428] T b016bfa0fb474da1a6fc334def67deba P 63d442048614417ab0e8215eb33a4e59: Time spent starting tablet: real 0.010s user 0.006s sys 0.000s
I20260811 18:57:37.127472 20597 tablet_service.cc:1467] Processing CreateTablet for tablet b016bfa0fb474da1a6fc334def67deba (DEFAULT_TABLE table=test-table [id=702f0c03f0424078bdd479c4aab85da5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.128655 20597 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b016bfa0fb474da1a6fc334def67deba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.129070 20976 heartbeater.cc:503] Master 127.27.180.190:33551 was elected leader, sending a full tablet report...
I20260811 18:57:37.135245 20337 tablet_service.cc:1467] Processing CreateTablet for tablet b016bfa0fb474da1a6fc334def67deba (DEFAULT_TABLE table=test-table [id=702f0c03f0424078bdd479c4aab85da5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:37.136312 20337 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b016bfa0fb474da1a6fc334def67deba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:37.144771 21034 tablet_bootstrap.cc:492] T b016bfa0fb474da1a6fc334def67deba P faa4fad679144835a25568d63b2cd2ee: Bootstrap starting.
I20260811 18:57:37.147136 21034 tablet_bootstrap.cc:654] T b016bfa0fb474da1a6fc334def67deba P faa4fad679144835a25568d63b2cd2ee: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:37.147536 21040 tablet_bootstrap.cc:492] T b016bfa0fb474da1a6fc334def67deba P aeb897b1c02c482ba7b4ec29b3d28484: Bootstrap starting.
I20260811 18:57:37.148015 21034 log.cc:824] T b016bfa0fb474da1a6fc334def67deba P faa4fad679144835a25568d63b2cd2ee: Log is configured to *not* fsync() on all Append() calls
I20260811 18:57:37.149937 21040 tablet_bootstrap.cc:654] T b016bfa0fb474da1a6fc334def67deba P aeb897b1c02c482ba7b4ec29b3d28484: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:37.150678 21040 log.cc:824] T b016bfa0fb474da1a6fc334def67deba P aeb897b1c02c482ba7b4ec29b3d28484: Log is configured to *not* fsync() on all Append() calls
I20260811 18:57:37.150827 21034 tablet_bootstrap.cc:492] T b016bfa0fb474da1a6fc334def67deba P faa4fad679144835a25568d63b2cd2ee: No bootstrap required, opened a new log
I20260811 18:57:37.151036 21034 ts_tablet_manager.cc:1397] T b016bfa0fb474da1a6fc334def67deba P faa4fad679144835a25568d63b2cd2ee: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 18:57:37.154980 21034 raft_consensus.cc:348] T b016bfa0fb474da1a6fc334def67deba P faa4fad679144835a25568d63b2cd2ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aeb897b1c02c482ba7b4ec29b3d28484" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 41129 } } peers { permanent_uuid: "63d442048614417ab0e8215eb33a4e59" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 39189 } } peers { permanent_uuid: "faa4fad679144835a25568d63b2cd2ee" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 35901 } }
I20260811 18:57:37.155252 21034 raft_consensus.cc:374] T b016bfa0fb474da1a6fc334def67deba P faa4fad679144835a25568d63b2cd2ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:37.155297 21034 raft_consensus.cc:729] T b016bfa0fb474da1a6fc334def67deba P faa4fad679144835a25568d63b2cd2ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: faa4fad679144835a25568d63b2cd2ee, State: Initialized, Role: FOLLOWER
I20260811 18:57:37.155781 21034 consensus_queue.cc:260] T b016bfa0fb474da1a6fc334def67deba P faa4fad679144835a25568d63b2cd2ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aeb897b1c02c482ba7b4ec29b3d28484" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 41129 } } peers { permanent_uuid: "63d442048614417ab0e8215eb33a4e59" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 39189 } } peers { permanent_uuid: "faa4fad679144835a25568d63b2cd2ee" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 35901 } }
I20260811 18:57:37.156807 21034 ts_tablet_manager.cc:1428] T b016bfa0fb474da1a6fc334def67deba P faa4fad679144835a25568d63b2cd2ee: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 18:57:37.157158 20465 heartbeater.cc:503] Master 127.27.180.190:33551 was elected leader, sending a full tablet report...
I20260811 18:57:37.157441 21040 tablet_bootstrap.cc:492] T b016bfa0fb474da1a6fc334def67deba P aeb897b1c02c482ba7b4ec29b3d28484: No bootstrap required, opened a new log
I20260811 18:57:37.157644 21040 ts_tablet_manager.cc:1397] T b016bfa0fb474da1a6fc334def67deba P aeb897b1c02c482ba7b4ec29b3d28484: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.002s
I20260811 18:57:37.165069 21040 raft_consensus.cc:348] T b016bfa0fb474da1a6fc334def67deba P aeb897b1c02c482ba7b4ec29b3d28484 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aeb897b1c02c482ba7b4ec29b3d28484" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 41129 } } peers { permanent_uuid: "63d442048614417ab0e8215eb33a4e59" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 39189 } } peers { permanent_uuid: "faa4fad679144835a25568d63b2cd2ee" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 35901 } }
I20260811 18:57:37.165788 21040 raft_consensus.cc:374] T b016bfa0fb474da1a6fc334def67deba P aeb897b1c02c482ba7b4ec29b3d28484 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:37.166079 21040 raft_consensus.cc:729] T b016bfa0fb474da1a6fc334def67deba P aeb897b1c02c482ba7b4ec29b3d28484 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aeb897b1c02c482ba7b4ec29b3d28484, State: Initialized, Role: FOLLOWER
I20260811 18:57:37.168160 21040 consensus_queue.cc:260] T b016bfa0fb474da1a6fc334def67deba P aeb897b1c02c482ba7b4ec29b3d28484 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aeb897b1c02c482ba7b4ec29b3d28484" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 41129 } } peers { permanent_uuid: "63d442048614417ab0e8215eb33a4e59" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 39189 } } peers { permanent_uuid: "faa4fad679144835a25568d63b2cd2ee" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 35901 } }
I20260811 18:57:37.169461 21040 ts_tablet_manager.cc:1428] T b016bfa0fb474da1a6fc334def67deba P aeb897b1c02c482ba7b4ec29b3d28484: Time spent starting tablet: real 0.012s user 0.003s sys 0.004s
I20260811 18:57:37.169857 20730 heartbeater.cc:503] Master 127.27.180.190:33551 was elected leader, sending a full tablet report...
W20260811 18:57:37.249040 20470 tablet.cc:2369] T b016bfa0fb474da1a6fc334def67deba P faa4fad679144835a25568d63b2cd2ee: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 18:57:37.254009 20731 tablet.cc:2369] T b016bfa0fb474da1a6fc334def67deba P aeb897b1c02c482ba7b4ec29b3d28484: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 18:57:37.262395 20977 tablet.cc:2369] T b016bfa0fb474da1a6fc334def67deba P 63d442048614417ab0e8215eb33a4e59: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:57:37.350822 21045 raft_consensus.cc:482] T b016bfa0fb474da1a6fc334def67deba P faa4fad679144835a25568d63b2cd2ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:37.351094 21045 raft_consensus.cc:504] T b016bfa0fb474da1a6fc334def67deba P faa4fad679144835a25568d63b2cd2ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aeb897b1c02c482ba7b4ec29b3d28484" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 41129 } } peers { permanent_uuid: "63d442048614417ab0e8215eb33a4e59" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 39189 } } peers { permanent_uuid: "faa4fad679144835a25568d63b2cd2ee" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 35901 } }
I20260811 18:57:37.352581 21045 leader_election.cc:288] T b016bfa0fb474da1a6fc334def67deba P faa4fad679144835a25568d63b2cd2ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aeb897b1c02c482ba7b4ec29b3d28484 (127.27.180.130:41129), 63d442048614417ab0e8215eb33a4e59 (127.27.180.131:39189)
I20260811 18:57:37.358361 21021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:37.353246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.129:39077 (local address 127.27.180.131:39189)
0811 18:57:37.353374 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:37.353379 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:37.353591 (+ 212us) server_negotiation.cc:407] Connection header received
0811 18:57:37.353777 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:37.353780 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:37.353828 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:37.353884 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:57:37.354434 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:37.355242 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:37.357039 (+ 1797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:37.357617 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:37.357717 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:37.357772 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:37.358148 (+ 376us) server_negotiation.cc:299] Negotiation successful
0811 18:57:37.358232 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 18:57:37.358397 21054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:37.352775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.130:41129 (local address 127.27.180.129:57625)
0811 18:57:37.353461 (+ 686us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:37.353474 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:37.353645 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:37.354020 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:37.354023 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:37.354062 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:57:37.354293 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:37.354301 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:37.355390 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:37.355393 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:37.356947 (+ 1554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:37.356964 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:37.357957 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:37.357961 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:37.358068 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:37.358092 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 18:57:37.358156 (+ 64us) client_negotiation.cc:240] Negotiation successful
0811 18:57:37.358227 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":614,"mutex_wait_us":235,"spinlock_wait_cycles":7936,"thread_start_us":107,"threads_started":1}
I20260811 18:57:37.358806 21053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:37.352882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.131:39189 (local address 127.27.180.129:39077)
0811 18:57:37.353521 (+ 639us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:37.353531 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:37.353664 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:37.354020 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:37.354023 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:37.354060 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:57:37.354293 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:37.354301 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:37.355390 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:37.355393 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:37.356885 (+ 1492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:37.356899 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:37.357891 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:37.357894 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:37.357995 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:37.358020 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 18:57:37.358568 (+ 548us) client_negotiation.cc:240] Negotiation successful
0811 18:57:37.358647 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":589,"mutex_wait_us":126,"spinlock_wait_cycles":64256,"thread_start_us":85,"threads_started":1}
I20260811 18:57:37.358875 21023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:37.353253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.129:57625 (local address 127.27.180.130:41129)
0811 18:57:37.353400 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:37.353405 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:37.353555 (+ 150us) server_negotiation.cc:407] Connection header received
0811 18:57:37.353776 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:37.353779 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:37.353827 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:37.353876 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:57:37.354436 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:37.355242 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:37.357143 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:37.357721 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:37.357846 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:37.357896 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:37.358668 (+ 772us) server_negotiation.cc:299] Negotiation successful
0811 18:57:37.358759 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 18:57:37.359426 20893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b016bfa0fb474da1a6fc334def67deba" candidate_uuid: "faa4fad679144835a25568d63b2cd2ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63d442048614417ab0e8215eb33a4e59" is_pre_election: true
I20260811 18:57:37.359431 20639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b016bfa0fb474da1a6fc334def67deba" candidate_uuid: "faa4fad679144835a25568d63b2cd2ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aeb897b1c02c482ba7b4ec29b3d28484" is_pre_election: true
I20260811 18:57:37.359723 20893 raft_consensus.cc:2454] T b016bfa0fb474da1a6fc334def67deba P 63d442048614417ab0e8215eb33a4e59 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate faa4fad679144835a25568d63b2cd2ee in term 0.
I20260811 18:57:37.359737 20639 raft_consensus.cc:2454] T b016bfa0fb474da1a6fc334def67deba P aeb897b1c02c482ba7b4ec29b3d28484 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate faa4fad679144835a25568d63b2cd2ee in term 0.
I20260811 18:57:37.360214 20255 leader_election.cc:302] T b016bfa0fb474da1a6fc334def67deba P faa4fad679144835a25568d63b2cd2ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 63d442048614417ab0e8215eb33a4e59, aeb897b1c02c482ba7b4ec29b3d28484, faa4fad679144835a25568d63b2cd2ee; no voters:
I20260811 18:57:37.360496 21045 raft_consensus.cc:2781] T b016bfa0fb474da1a6fc334def67deba P faa4fad679144835a25568d63b2cd2ee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:37.360576 21045 raft_consensus.cc:482] T b016bfa0fb474da1a6fc334def67deba P faa4fad679144835a25568d63b2cd2ee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:37.360616 21045 raft_consensus.cc:3037] T b016bfa0fb474da1a6fc334def67deba P faa4fad679144835a25568d63b2cd2ee [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:37.361982 21045 raft_consensus.cc:504] T b016bfa0fb474da1a6fc334def67deba P faa4fad679144835a25568d63b2cd2ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aeb897b1c02c482ba7b4ec29b3d28484" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 41129 } } peers { permanent_uuid: "63d442048614417ab0e8215eb33a4e59" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 39189 } } peers { permanent_uuid: "faa4fad679144835a25568d63b2cd2ee" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 35901 } }
I20260811 18:57:37.362315 21045 leader_election.cc:288] T b016bfa0fb474da1a6fc334def67deba P faa4fad679144835a25568d63b2cd2ee [CANDIDATE]: Term 1 election: Requested vote from peers aeb897b1c02c482ba7b4ec29b3d28484 (127.27.180.130:41129), 63d442048614417ab0e8215eb33a4e59 (127.27.180.131:39189)
I20260811 18:57:37.362637 20639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b016bfa0fb474da1a6fc334def67deba" candidate_uuid: "faa4fad679144835a25568d63b2cd2ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aeb897b1c02c482ba7b4ec29b3d28484"
I20260811 18:57:37.362687 20893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b016bfa0fb474da1a6fc334def67deba" candidate_uuid: "faa4fad679144835a25568d63b2cd2ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63d442048614417ab0e8215eb33a4e59"
I20260811 18:57:37.362761 20639 raft_consensus.cc:3037] T b016bfa0fb474da1a6fc334def67deba P aeb897b1c02c482ba7b4ec29b3d28484 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:37.362804 20893 raft_consensus.cc:3037] T b016bfa0fb474da1a6fc334def67deba P 63d442048614417ab0e8215eb33a4e59 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:37.363855 20893 raft_consensus.cc:2454] T b016bfa0fb474da1a6fc334def67deba P 63d442048614417ab0e8215eb33a4e59 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate faa4fad679144835a25568d63b2cd2ee in term 1.
I20260811 18:57:37.363996 20639 raft_consensus.cc:2454] T b016bfa0fb474da1a6fc334def67deba P aeb897b1c02c482ba7b4ec29b3d28484 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate faa4fad679144835a25568d63b2cd2ee in term 1.
I20260811 18:57:37.364174 20255 leader_election.cc:302] T b016bfa0fb474da1a6fc334def67deba P faa4fad679144835a25568d63b2cd2ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 63d442048614417ab0e8215eb33a4e59, faa4fad679144835a25568d63b2cd2ee; no voters:
I20260811 18:57:37.364413 21045 raft_consensus.cc:2781] T b016bfa0fb474da1a6fc334def67deba P faa4fad679144835a25568d63b2cd2ee [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:37.364638 21045 raft_consensus.cc:686] T b016bfa0fb474da1a6fc334def67deba P faa4fad679144835a25568d63b2cd2ee [term 1 LEADER]: Becoming Leader. State: Replica: faa4fad679144835a25568d63b2cd2ee, State: Running, Role: LEADER
I20260811 18:57:37.364970 21045 consensus_queue.cc:237] T b016bfa0fb474da1a6fc334def67deba P faa4fad679144835a25568d63b2cd2ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aeb897b1c02c482ba7b4ec29b3d28484" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 41129 } } peers { permanent_uuid: "63d442048614417ab0e8215eb33a4e59" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 39189 } } peers { permanent_uuid: "faa4fad679144835a25568d63b2cd2ee" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 35901 } }
I20260811 18:57:37.368341 20095 catalog_manager.cc:5495] T b016bfa0fb474da1a6fc334def67deba P faa4fad679144835a25568d63b2cd2ee reported cstate change: term changed from 0 to 1, leader changed from <none> to faa4fad679144835a25568d63b2cd2ee (127.27.180.129). New cstate: current_term: 1 leader_uuid: "faa4fad679144835a25568d63b2cd2ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aeb897b1c02c482ba7b4ec29b3d28484" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 41129 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "63d442048614417ab0e8215eb33a4e59" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 39189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "faa4fad679144835a25568d63b2cd2ee" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 35901 } health_report { overall_health: HEALTHY } } }
I20260811 18:57:37.397851 21027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:37.393866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33886 (local address 127.27.180.129:35901)
0811 18:57:37.394027 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:37.394031 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:37.394048 (+ 17us) server_negotiation.cc:407] Connection header received
0811 18:57:37.394135 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:37.394138 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:37.394204 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:37.394315 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:57:37.394590 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:37.395334 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:37.396198 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:37.396532 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:37.396634 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:37.396873 (+ 239us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:57:37.397437 (+ 564us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:57:37.397553 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:37.397620 (+ 67us) server_negotiation.cc:299] Negotiation successful
0811 18:57:37.397680 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 18:57:37.404239 20893 raft_consensus.cc:1261] T b016bfa0fb474da1a6fc334def67deba P 63d442048614417ab0e8215eb33a4e59 [term 1 FOLLOWER]: Refusing update from remote peer faa4fad679144835a25568d63b2cd2ee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:57:37.404239 20639 raft_consensus.cc:1261] T b016bfa0fb474da1a6fc334def67deba P aeb897b1c02c482ba7b4ec29b3d28484 [term 1 FOLLOWER]: Refusing update from remote peer faa4fad679144835a25568d63b2cd2ee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:57:37.405025 21045 consensus_queue.cc:1035] T b016bfa0fb474da1a6fc334def67deba P faa4fad679144835a25568d63b2cd2ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "63d442048614417ab0e8215eb33a4e59" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 39189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:37.405537 21063 consensus_queue.cc:1035] T b016bfa0fb474da1a6fc334def67deba P faa4fad679144835a25568d63b2cd2ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "aeb897b1c02c482ba7b4ec29b3d28484" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 41129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:37.414254 21071 mvcc.cc:205] Tried to move back new op lower bound from 7317400196719992832 to 7317400196569346048. Current Snapshot: MvccSnapshot[applied={T|T < 7317400196719992832}]
I20260811 18:57:37.415947 21079 mvcc.cc:205] Tried to move back new op lower bound from 7317400196719992832 to 7317400196569346048. Current Snapshot: MvccSnapshot[applied={T|T < 7317400196719992832}]
I20260811 18:57:37.416126 21077 mvcc.cc:205] Tried to move back new op lower bound from 7317400196719992832 to 7317400196569346048. Current Snapshot: MvccSnapshot[applied={T|T < 7317400196719992832}]
I20260811 18:57:37.425062 20094 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47344:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:57:37.425256 20094 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47344:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:57:37.427902 20094 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7c3b1394721845259008d27a9a7ff684: Sending DeleteTablet for 3 replicas of tablet b016bfa0fb474da1a6fc334def67deba
I20260811 18:57:37.428838 20337 tablet_service.cc:1514] Processing DeleteTablet for tablet b016bfa0fb474da1a6fc334def67deba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:37 UTC) from {username='kudu', principal='kudu/127.27.180.190@KRBTEST.COM'} at 127.0.0.1:33872
I20260811 18:57:37.428846 20860 tablet_service.cc:1514] Processing DeleteTablet for tablet b016bfa0fb474da1a6fc334def67deba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:37 UTC) from {username='kudu', principal='kudu/127.27.180.190@KRBTEST.COM'} at 127.0.0.1:51456
I20260811 18:57:37.429515 21092 tablet_replica.cc:330] T b016bfa0fb474da1a6fc334def67deba P faa4fad679144835a25568d63b2cd2ee: stopping tablet replica
I20260811 18:57:37.429692 20597 tablet_service.cc:1514] Processing DeleteTablet for tablet b016bfa0fb474da1a6fc334def67deba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:37 UTC) from {username='kudu', principal='kudu/127.27.180.190@KRBTEST.COM'} at 127.0.0.1:39120
I20260811 18:57:37.429973 21092 raft_consensus.cc:2229] T b016bfa0fb474da1a6fc334def67deba P faa4fad679144835a25568d63b2cd2ee [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:37.430367 21092 raft_consensus.cc:2258] T b016bfa0fb474da1a6fc334def67deba P faa4fad679144835a25568d63b2cd2ee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:37.430831 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 20167
I20260811 18:57:37.443435 21095 tablet_replica.cc:330] T b016bfa0fb474da1a6fc334def67deba P aeb897b1c02c482ba7b4ec29b3d28484: stopping tablet replica
I20260811 18:57:37.443845 21095 raft_consensus.cc:2229] T b016bfa0fb474da1a6fc334def67deba P aeb897b1c02c482ba7b4ec29b3d28484 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:37.444229 21095 raft_consensus.cc:2258] T b016bfa0fb474da1a6fc334def67deba P aeb897b1c02c482ba7b4ec29b3d28484 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:37.444661 21093 tablet_replica.cc:330] T b016bfa0fb474da1a6fc334def67deba P 63d442048614417ab0e8215eb33a4e59: stopping tablet replica
I20260811 18:57:37.444968 21093 raft_consensus.cc:2229] T b016bfa0fb474da1a6fc334def67deba P 63d442048614417ab0e8215eb33a4e59 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:37.445333 21093 raft_consensus.cc:2258] T b016bfa0fb474da1a6fc334def67deba P 63d442048614417ab0e8215eb33a4e59 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:37.446269 21095 ts_tablet_manager.cc:1905] T b016bfa0fb474da1a6fc334def67deba P aeb897b1c02c482ba7b4ec29b3d28484: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:37.446624 21093 ts_tablet_manager.cc:1905] T b016bfa0fb474da1a6fc334def67deba P 63d442048614417ab0e8215eb33a4e59: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:37.449142 21095 ts_tablet_manager.cc:1918] T b016bfa0fb474da1a6fc334def67deba P aeb897b1c02c482ba7b4ec29b3d28484: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:57:37.449282 21093 ts_tablet_manager.cc:1918] T b016bfa0fb474da1a6fc334def67deba P 63d442048614417ab0e8215eb33a4e59: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:57:37.449309 21095 log.cc:1196] T b016bfa0fb474da1a6fc334def67deba P aeb897b1c02c482ba7b4ec29b3d28484: Deleting WAL directory at /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ts-1/wal/wals/b016bfa0fb474da1a6fc334def67deba
I20260811 18:57:37.449379 21093 log.cc:1196] T b016bfa0fb474da1a6fc334def67deba P 63d442048614417ab0e8215eb33a4e59: Deleting WAL directory at /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786474496157106-28370-0/minicluster-data/ts-2/wal/wals/b016bfa0fb474da1a6fc334def67deba
I20260811 18:57:37.449673 21093 ts_tablet_manager.cc:1939] T b016bfa0fb474da1a6fc334def67deba P 63d442048614417ab0e8215eb33a4e59: Deleting consensus metadata
I20260811 18:57:37.449808 21095 ts_tablet_manager.cc:1939] T b016bfa0fb474da1a6fc334def67deba P aeb897b1c02c482ba7b4ec29b3d28484: Deleting consensus metadata
W20260811 18:57:37.450917 20060 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.27.180.129:35901 (error 108)
I20260811 18:57:37.450934 20071 catalog_manager.cc:4841] TS 63d442048614417ab0e8215eb33a4e59 (127.27.180.131:39189): tablet b016bfa0fb474da1a6fc334def67deba (table test-table [id=702f0c03f0424078bdd479c4aab85da5]) successfully deleted
I20260811 18:57:37.450968 20069 catalog_manager.cc:4841] TS aeb897b1c02c482ba7b4ec29b3d28484 (127.27.180.130:41129): tablet b016bfa0fb474da1a6fc334def67deba (table test-table [id=702f0c03f0424078bdd479c4aab85da5]) successfully deleted
I20260811 18:57:37.451808 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 20475
I20260811 18:57:37.452106 21020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:37.451788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.129:35901 (local address 127.0.0.1:33888)
0811 18:57:37.451958 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:37.452025 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.129:35901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
W20260811 18:57:37.454777 20060 catalog_manager.cc:4584] TS faa4fad679144835a25568d63b2cd2ee (127.27.180.129:35901): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b016bfa0fb474da1a6fc334def67deba: Network error: Client connection negotiation failed: client connection to 127.27.180.129:35901: connect: Connection refused (error 111)
I20260811 18:57:37.464389 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 20733
I20260811 18:57:37.480670 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 20021
I20260811 18:57:37.495436 28370 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260811 18:57:37.840070 28370 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260811 18:57:37.840171 28370 mini_ranger.cc:66] Stopping Ranger...
I20260811 18:57:38.185150 28370 mini_ranger.cc:68] Stopped Ranger
2026-08-11 18:57:38.185 UTC [18283] LOG: received smart shutdown request
2026-08-11 18:57:38.189 UTC [18283] LOG: background worker "logical replication launcher" (PID 18295) exited with exit code 1
2026-08-11 18:57:38.189 UTC [18288] LOG: shutting down
2026-08-11 18:57:38.339 UTC [18283] LOG: database system is shut down
2026-08-11T18:57:38Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (49220 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474496157106-28370-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:57:38 dist-test-slave-2j7r krb5kdc[21115](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:57:38 dist-test-slave-2j7r krb5kdc[21115](info): set up 2 sockets
Aug 11 18:57:38 dist-test-slave-2j7r krb5kdc[21115](info): commencing operation
krb5kdc: starting...
W20260811 18:57:39.636996 28370 mini_kdc.cc:121] Time spent starting KDC: real 1.173s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474496157106-28370-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474496157106-28370-0/krb5kdc/test-user.keytab.
Aug 11 18:57:39 dist-test-slave-2j7r krb5kdc[21115](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474659, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:57:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:57:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.180.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.180.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.180.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab.
Entry for principal kudu/127.27.180.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab.
I20260811 18:57:39.757984 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474496157106-28370-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474496157106-28370-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474496157106-28370-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.190:32983
--webserver_interface=127.27.180.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab
--principal=kudu/127.27.180.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:36585
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.180.190:32983,127.27.180.189:32801,127.27.180.188:35091
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:57:39.866667 21135 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:57:39.866899 21135 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:57:39.866946 21135 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:57:39.870828 21135 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:57:39.870877 21135 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:57:39.870894 21135 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:57:39.870908 21135 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:57:39.870926 21135 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:57:39.875936 21135 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:36585
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474496157106-28370-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474496157106-28370-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.180.190:32983,127.27.180.189:32801,127.27.180.188:35091
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab
--principal=kudu/127.27.180.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.190:32983
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.180.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21135
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474496157106-28370-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:57:39.876857 21135 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:57:39.877811 21135 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:57:39.885697 21151 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:39.885689 21156 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:57:39.885764 21135 server_base.cc:1034] running on GCE node
W20260811 18:57:39.885673 21150 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:57:39.886487 21135 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:57:39.887503 21135 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:57:39.888657 21135 hybrid_clock.cc:648] HybridClock initialized: now 1786474659888632 us; error 38 us; skew 500 ppm
Aug 11 18:57:39 dist-test-slave-2j7r krb5kdc[21115](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474659, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:57:39.891247 21135 init.cc:377] Logged in from keytab as kudu/127.27.180.190@KRBTEST.COM (short username kudu)
I20260811 18:57:39.892058 21135 webserver.cc:460] Webserver started at http://127.27.180.190:36555/ using document root <none> and password file <none>
I20260811 18:57:39.892576 21135 fs_manager.cc:359] Metadata directory not provided
I20260811 18:57:39.892639 21135 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:57:39.892864 21135 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:57:39.894867 21135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474496157106-28370-0/minicluster-data/master-0/data/instance:
uuid: "5df196ec71174cdaa2615c8170eda9a5"
format_stamp: "Formatted at 2026-08-11 18:57:39 on dist-test-slave-2j7r"
server_key: "979be9a8eed6534a4b7a181bf8420de8"
server_key_iv: "b3d460dbcd9b685a4afc2e09be9b0a0b"
server_key_version: "encryptionkey@0"
I20260811 18:57:39.895401 21135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474496157106-28370-0/minicluster-data/master-0/wal/instance:
uuid: "5df196ec71174cdaa2615c8170eda9a5"
format_stamp: "Formatted at 2026-08-11 18:57:39 on dist-test-slave-2j7r"
server_key: "979be9a8eed6534a4b7a181bf8420de8"
server_key_iv: "b3d460dbcd9b685a4afc2e09be9b0a0b"
server_key_version: "encryptionkey@0"
I20260811 18:57:39.898902 21135 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 18:57:39.901167 21166 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:39.902344 21135 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 18:57:39.902454 21135 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474496157106-28370-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474496157106-28370-0/minicluster-data/master-0/wal
uuid: "5df196ec71174cdaa2615c8170eda9a5"
format_stamp: "Formatted at 2026-08-11 18:57:39 on dist-test-slave-2j7r"
server_key: "979be9a8eed6534a4b7a181bf8420de8"
server_key_iv: "b3d460dbcd9b685a4afc2e09be9b0a0b"
server_key_version: "encryptionkey@0"
I20260811 18:57:39.902542 21135 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474496157106-28370-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474496157106-28370-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474496157106-28370-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:57:39.945848 21135 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:57:39.948827 21135 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:57:39.949009 21135 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:57:39.956246 21135 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.190:32983
I20260811 18:57:39.956521 21268 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.190:32983 every 8 connection(s)
I20260811 18:57:39.957255 21135 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
I20260811 18:57:39.960127 21269 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:39.963776 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 21135
I20260811 18:57:39.963919 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474496157106-28370-0/minicluster-data/master-0/wal/instance
I20260811 18:57:39.963814 21269 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.180.190" port: 32983 } has no permanent_uuid. Determining permanent_uuid...
I20260811 18:57:39.964176 28370 external_mini_cluster.cc:1428] Setting key bdb1c382c4fc796061503231d26827c2
WARNING: no policy specified for kudu/127.27.180.189@KRBTEST.COM; defaulting to no policy
Aug 11 18:57:39 dist-test-slave-2j7r krb5kdc[21115](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474659, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.190@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.180.189@KRBTEST.COM" created.
I20260811 18:57:39.978909 21274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:39.966166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60694 (local address 127.27.180.190:32983)
0811 18:57:39.966748 (+ 582us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:39.966760 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:39.966805 (+ 45us) server_negotiation.cc:407] Connection header received
0811 18:57:39.967807 (+ 1002us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:39.967813 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:39.967996 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:39.968152 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:39.969168 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:39.970030 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:39.971439 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:39.971805 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:39.971926 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:39.974355 (+ 2429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:39.974360 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:39.974369 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:39.974391 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:39.975834 (+ 1443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:39.976275 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:39.976280 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:39.976281 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:39.976450 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:39.976755 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:39.976759 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:39.976761 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:39.977154 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:39.977274 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:39.977625 (+ 351us) server_negotiation.cc:299] Negotiation successful
0811 18:57:39.977959 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":129,"server-negotiator.queue_time_us":468,"spinlock_wait_cycles":54144,"thread_start_us":185,"threads_started":1}
I20260811 18:57:39.978909 21272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:39.966167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:32983 (local address 127.0.0.1:60694)
0811 18:57:39.966668 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:39.966719 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:39.967527 (+ 808us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:39.968207 (+ 680us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:39.968213 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:39.968643 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:57:39.968975 (+ 332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:39.968993 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:39.970161 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:39.970165 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:39.971304 (+ 1139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:39.971316 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:39.971957 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:39.971961 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:39.972085 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:39.972610 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:57:39.972630 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:57:39.974185 (+ 1555us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:57:39.975935 (+ 1750us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:39.975943 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:39.975953 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:39.976173 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:39.976542 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:39.976546 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:39.976548 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:39.976661 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:39.977259 (+ 598us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:57:39.977266 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:57:39.977403 (+ 137us) client_negotiation.cc:769] Sending connection context
0811 18:57:39.977549 (+ 146us) client_negotiation.cc:240] Negotiation successful
0811 18:57:39.977894 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"mutex_wait_us":1,"thread_start_us":125,"threads_started":1}
I20260811 18:57:39.981809 21269 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.180.189" port: 32801 } has no permanent_uuid. Determining permanent_uuid...
I20260811 18:57:39.982640 21272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:39.982385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.189:32801 (local address 127.0.0.1:60078)
0811 18:57:39.982499 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:39.982571 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.189:32801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260811 18:57:39.983599 21171 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.27.180.189:32801: connect: Connection refused (error 111)
I20260811 18:57:39.984433 21272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:39.984232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.189:32801 (local address 127.0.0.1:60084)
0811 18:57:39.984332 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:39.984378 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.189:32801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260811 18:57:39.984555 21269 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.180.189:32801: Network error: Client connection negotiation failed: client connection to 127.27.180.189:32801: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.180.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.189.keytab.
Entry for principal kudu/127.27.180.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.189.keytab.
I20260811 18:57:39.991973 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474496157106-28370-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474496157106-28370-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474496157106-28370-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474496157106-28370-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.189:32801
--webserver_interface=127.27.180.189
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.189.keytab
--principal=kudu/127.27.180.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:36585
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.180.190:32983,127.27.180.189:32801,127.27.180.188:35091
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 18:57:40.033797 21269 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.180.189" port: 32801 } attempt: 1
I20260811 18:57:40.034643 21272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:40.034353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.189:32801 (local address 127.0.0.1:60098)
0811 18:57:40.034500 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:40.034564 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.189:32801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260811 18:57:40.035434 21272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:40.035217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.189:32801 (local address 127.0.0.1:60112)
0811 18:57:40.035324 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:40.035371 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.189:32801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260811 18:57:40.035549 21269 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.180.189:32801: Network error: Client connection negotiation failed: client connection to 127.27.180.189:32801: connect: Connection refused (error 111)
W20260811 18:57:40.101778 21277 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:57:40.102011 21277 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:57:40.102047 21277 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260811 18:57:40.103713 21269 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.180.189" port: 32801 } attempt: 2
I20260811 18:57:40.104559 21272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:40.104242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.189:32801 (local address 127.0.0.1:60124)
0811 18:57:40.104417 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:40.104480 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.189:32801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260811 18:57:40.105261 21272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:40.105023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.189:32801 (local address 127.0.0.1:60126)
0811 18:57:40.105130 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:40.105174 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.189:32801: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260811 18:57:40.105383 21269 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.180.189:32801: Network error: Client connection negotiation failed: client connection to 127.27.180.189:32801: connect: Connection refused (error 111)
W20260811 18:57:40.106256 21277 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:57:40.106303 21277 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:57:40.106323 21277 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:57:40.106354 21277 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:57:40.106383 21277 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:57:40.111457 21277 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:36585
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474496157106-28370-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474496157106-28370-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.27.180.190:32983,127.27.180.189:32801,127.27.180.188:35091
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.189.keytab
--principal=kudu/127.27.180.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.189:32801
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474496157106-28370-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.27.180.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21277
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474496157106-28370-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:57:40.112380 21277 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:57:40.113353 21277 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:57:40.121294 21283 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:40.121310 21288 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:40.121330 21284 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:57:40.121543 21277 server_base.cc:1034] running on GCE node
I20260811 18:57:40.122145 21277 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:57:40.123152 21277 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:57:40.124326 21277 hybrid_clock.cc:648] HybridClock initialized: now 1786474660124294 us; error 31 us; skew 500 ppm
Aug 11 18:57:40 dist-test-slave-2j7r krb5kdc[21115](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474660, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:57:40.127082 21277 init.cc:377] Logged in from keytab as kudu/127.27.180.189@KRBTEST.COM (short username kudu)
I20260811 18:57:40.127926 21277 webserver.cc:460] Webserver started at http://127.27.180.189:46557/ using document root <none> and password file <none>
I20260811 18:57:40.128437 21277 fs_manager.cc:359] Metadata directory not provided
I20260811 18:57:40.128504 21277 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:57:40.128701 21277 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:57:40.130712 21277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474496157106-28370-0/minicluster-data/master-1/data/instance:
uuid: "c94f647e51864f6b924f300b69fc79eb"
format_stamp: "Formatted at 2026-08-11 18:57:40 on dist-test-slave-2j7r"
server_key: "6871b0a586f1afcfee89f999fb58f8c4"
server_key_iv: "f03ad7a0c1853d223fd1db721273608c"
server_key_version: "encryptionkey@0"
I20260811 18:57:40.131232 21277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474496157106-28370-0/minicluster-data/master-1/wal/instance:
uuid: "c94f647e51864f6b924f300b69fc79eb"
format_stamp: "Formatted at 2026-08-11 18:57:40 on dist-test-slave-2j7r"
server_key: "6871b0a586f1afcfee89f999fb58f8c4"
server_key_iv: "f03ad7a0c1853d223fd1db721273608c"
server_key_version: "encryptionkey@0"
I20260811 18:57:40.134701 21277 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.005s
I20260811 18:57:40.136933 21310 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:40.138156 21277 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260811 18:57:40.138267 21277 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474496157106-28370-0/minicluster-data/master-1/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474496157106-28370-0/minicluster-data/master-1/wal
uuid: "c94f647e51864f6b924f300b69fc79eb"
format_stamp: "Formatted at 2026-08-11 18:57:40 on dist-test-slave-2j7r"
server_key: "6871b0a586f1afcfee89f999fb58f8c4"
server_key_iv: "f03ad7a0c1853d223fd1db721273608c"
server_key_version: "encryptionkey@0"
I20260811 18:57:40.138376 21277 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474496157106-28370-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474496157106-28370-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474496157106-28370-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:57:40.151433 21277 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:57:40.154187 21277 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:57:40.154357 21277 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:57:40.161327 21277 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.189:32801
I20260811 18:57:40.161667 21404 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.189:32801 every 8 connection(s)
I20260811 18:57:40.162292 21277 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474496157106-28370-0/minicluster-data/master-1/data/info.pb
I20260811 18:57:40.164999 21405 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:40.167877 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 21277
I20260811 18:57:40.168022 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474496157106-28370-0/minicluster-data/master-1/wal/instance
I20260811 18:57:40.168326 28370 external_mini_cluster.cc:1428] Setting key 425b9a8facdb85e5c4a3d3b3d172d2ee
I20260811 18:57:40.168957 21405 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.180.190" port: 32983 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.27.180.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.180.188@KRBTEST.COM" created.
Aug 11 18:57:40 dist-test-slave-2j7r krb5kdc[21115](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474660, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.189@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM
I20260811 18:57:40.185252 21274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:40.171142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60708 (local address 127.27.180.190:32983)
0811 18:57:40.171269 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:40.171273 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:40.172257 (+ 984us) server_negotiation.cc:407] Connection header received
0811 18:57:40.173177 (+ 920us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:40.173181 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:40.173257 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:40.173329 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:40.174446 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:40.174981 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:40.176094 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:40.176337 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:40.176431 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:40.180912 (+ 4481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:40.180916 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:40.180919 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:40.180951 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:40.182927 (+ 1976us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:40.183547 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:40.183550 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:40.183551 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:40.183628 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:40.184011 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:40.184013 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:40.184014 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:40.184215 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:40.184300 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:40.184923 (+ 623us) server_negotiation.cc:299] Negotiation successful
0811 18:57:40.185056 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 18:57:40.185981 21412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:40.171589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:32983 (local address 127.0.0.1:60708)
0811 18:57:40.172109 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:40.172149 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:40.172956 (+ 807us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:40.173497 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:40.173503 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:40.173834 (+ 331us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:57:40.174295 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:40.174310 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:40.175111 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:40.175114 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:40.175967 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:40.175977 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:40.176480 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:40.176485 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:40.176651 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:40.177307 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:57:40.177328 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:57:40.180734 (+ 3406us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:57:40.183045 (+ 2311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:40.183052 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:40.183065 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:40.183440 (+ 375us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:40.183705 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:40.183708 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:40.183710 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:40.183923 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:40.184322 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:57:40.184329 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:57:40.184645 (+ 316us) client_negotiation.cc:769] Sending connection context
0811 18:57:40.184837 (+ 192us) client_negotiation.cc:240] Negotiation successful
0811 18:57:40.185118 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":142,"threads_started":1}
I20260811 18:57:40.187417 21405 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.180.189" port: 32801 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 18:57:40 dist-test-slave-2j7r krb5kdc[21115](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474660, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.189@KRBTEST.COM for kudu/127.27.180.189@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.180.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.188.keytab.
Entry for principal kudu/127.27.180.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.188.keytab.
I20260811 18:57:40.196063 21412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:40.187927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.189:32801 (local address 127.0.0.1:60134)
0811 18:57:40.188107 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:40.188129 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:40.188205 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:40.188756 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:40.188759 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:40.188882 (+ 123us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:57:40.188966 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:40.188971 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:40.190003 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:40.190006 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:40.190567 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:40.190577 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:40.191159 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:40.191162 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:40.191241 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:40.191646 (+ 405us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:57:40.191657 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:57:40.192824 (+ 1167us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:57:40.194498 (+ 1674us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:40.194501 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:40.194502 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:40.194762 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:40.195043 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:40.195046 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:40.195047 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:40.195137 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:40.195575 (+ 438us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:57:40.195577 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:57:40.195679 (+ 102us) client_negotiation.cc:769] Sending connection context
0811 18:57:40.195785 (+ 106us) client_negotiation.cc:240] Negotiation successful
0811 18:57:40.195909 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116}
I20260811 18:57:40.196201 21414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:40.188046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60134 (local address 127.27.180.189:32801)
0811 18:57:40.188400 (+ 354us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:40.188412 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:40.188436 (+ 24us) server_negotiation.cc:407] Connection header received
0811 18:57:40.188479 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:40.188485 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:40.188636 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:40.188752 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:40.189145 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:40.189892 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:40.190679 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:40.191061 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:40.191144 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:40.192964 (+ 1820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:40.192974 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:40.192983 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:40.193004 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:40.194396 (+ 1392us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:40.194857 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:40.194862 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:40.194863 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:40.194938 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:40.195240 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:40.195242 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:40.195244 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:40.195499 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:40.195600 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:40.195889 (+ 289us) server_negotiation.cc:299] Negotiation successful
0811 18:57:40.196025 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":255,"thread_start_us":175,"threads_started":1}
I20260811 18:57:40.196513 21269 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.180.189" port: 32801 } attempt: 3
I20260811 18:57:40.196849 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474496157106-28370-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474496157106-28370-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474496157106-28370-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474496157106-28370-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.188:35091
--webserver_interface=127.27.180.188
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.188.keytab
--principal=kudu/127.27.180.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:36585
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.180.190:32983,127.27.180.189:32801,127.27.180.188:35091
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 18:57:40.198568 21405 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.180.188" port: 35091 } has no permanent_uuid. Determining permanent_uuid...
I20260811 18:57:40.199332 21412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:40.199062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.188:35091 (local address 127.0.0.1:55756)
0811 18:57:40.199182 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:40.199261 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.188:35091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 18:57:40.199527 21315 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.27.180.188:35091: connect: Connection refused (error 111)
I20260811 18:57:40.200636 21412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:40.200211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.188:35091 (local address 127.0.0.1:55758)
0811 18:57:40.200539 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:40.200580 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.188:35091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":270}
W20260811 18:57:40.200815 21405 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.180.188:35091: Network error: Client connection negotiation failed: client connection to 127.27.180.188:35091: connect: Connection refused (error 111)
Aug 11 18:57:40 dist-test-slave-2j7r krb5kdc[21115](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474659, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.190@KRBTEST.COM for kudu/127.27.180.189@KRBTEST.COM
I20260811 18:57:40.205633 21272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:40.196914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.189:32801 (local address 127.0.0.1:60140)
0811 18:57:40.197269 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:40.197283 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:40.197369 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:40.197617 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:40.197620 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:40.197807 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:57:40.197923 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:40.197930 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:40.198852 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:40.198856 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:40.199609 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:40.199622 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:40.200209 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:40.200213 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:40.200319 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:40.200751 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:57:40.200766 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:57:40.202286 (+ 1520us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:57:40.204162 (+ 1876us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:40.204165 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:40.204167 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:40.204371 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:40.204633 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:40.204635 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:40.204637 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:40.204703 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:40.205171 (+ 468us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:57:40.205174 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:57:40.205321 (+ 147us) client_negotiation.cc:769] Sending connection context
0811 18:57:40.205409 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 18:57:40.205519 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279}
I20260811 18:57:40.205740 21414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:40.197138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60140 (local address 127.27.180.189:32801)
0811 18:57:40.197246 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:40.197249 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:40.197324 (+ 75us) server_negotiation.cc:407] Connection header received
0811 18:57:40.197445 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:40.197448 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:40.197490 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:40.197563 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:40.198040 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:40.198709 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:40.199757 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:40.200087 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:40.200181 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:40.202429 (+ 2248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:40.202433 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:40.202436 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:40.202462 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:40.204045 (+ 1583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:40.204462 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:40.204466 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:40.204468 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:40.204553 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:40.204782 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:40.204785 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:40.204787 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:40.205002 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:40.205120 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:40.205486 (+ 366us) server_negotiation.cc:299] Negotiation successful
0811 18:57:40.205602 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 18:57:40.206861 21269 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.180.188" port: 35091 } has no permanent_uuid. Determining permanent_uuid...
I20260811 18:57:40.207561 21272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:40.207301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.188:35091 (local address 127.0.0.1:55772)
0811 18:57:40.207424 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:40.207482 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.188:35091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260811 18:57:40.208348 21272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:40.208063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.188:35091 (local address 127.0.0.1:55776)
0811 18:57:40.208230 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:40.208283 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.188:35091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
W20260811 18:57:40.208492 21269 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.180.188:35091: Network error: Client connection negotiation failed: client connection to 127.27.180.188:35091: connect: Connection refused (error 111)
I20260811 18:57:40.239670 21269 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.180.188" port: 35091 } attempt: 1
I20260811 18:57:40.240636 21272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:40.240266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.188:35091 (local address 127.0.0.1:55784)
0811 18:57:40.240451 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:40.240530 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.188:35091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260811 18:57:40.241396 21272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:40.241152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.188:35091 (local address 127.0.0.1:55788)
0811 18:57:40.241269 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:40.241324 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.188:35091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260811 18:57:40.241535 21269 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.180.188:35091: Network error: Client connection negotiation failed: client connection to 127.27.180.188:35091: connect: Connection refused (error 111)
I20260811 18:57:40.250044 21405 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.180.188" port: 35091 } attempt: 1
I20260811 18:57:40.250713 21412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:40.250466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.188:35091 (local address 127.0.0.1:55804)
0811 18:57:40.250593 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:40.250645 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.188:35091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260811 18:57:40.251412 21412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:40.251205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.188:35091 (local address 127.0.0.1:55820)
0811 18:57:40.251305 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:40.251354 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.188:35091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260811 18:57:40.251546 21405 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.180.188:35091: Network error: Client connection negotiation failed: client connection to 127.27.180.188:35091: connect: Connection refused (error 111)
W20260811 18:57:40.308853 21415 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:57:40.309157 21415 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:57:40.309199 21415 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:57:40.313359 21415 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:57:40.313416 21415 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:57:40.313433 21415 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:57:40.313447 21415 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:57:40.313460 21415 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:57:40.316730 21269 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.180.188" port: 35091 } attempt: 2
I20260811 18:57:40.317574 21272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:40.317261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.188:35091 (local address 127.0.0.1:55836)
0811 18:57:40.317416 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:40.317484 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.188:35091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260811 18:57:40.318256 21272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:40.318054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.188:35091 (local address 127.0.0.1:55840)
0811 18:57:40.318152 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:40.318199 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.188:35091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260811 18:57:40.318385 21269 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.180.188:35091: Network error: Client connection negotiation failed: client connection to 127.27.180.188:35091: connect: Connection refused (error 111)
I20260811 18:57:40.318899 21415 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:36585
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474496157106-28370-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474496157106-28370-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.27.180.190:32983,127.27.180.189:32801,127.27.180.188:35091
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.188.keytab
--principal=kudu/127.27.180.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.188:35091
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474496157106-28370-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.27.180.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21415
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474496157106-28370-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:57:40.319686 21405 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.180.188" port: 35091 } attempt: 2
I20260811 18:57:40.319840 21415 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:57:40.320320 21412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:40.320087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.188:35091 (local address 127.0.0.1:55856)
0811 18:57:40.320206 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:40.320256 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.188:35091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260811 18:57:40.320938 21412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:40.320757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.188:35091 (local address 127.0.0.1:55866)
0811 18:57:40.320836 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:40.320889 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.188:35091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260811 18:57:40.320983 21415 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:57:40.321048 21405 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.180.188:35091: Network error: Client connection negotiation failed: client connection to 127.27.180.188:35091: connect: Connection refused (error 111)
I20260811 18:57:40.329007 21415 server_base.cc:1034] running on GCE node
W20260811 18:57:40.328953 21429 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:40.328933 21423 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:40.328953 21427 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:57:40.329668 21415 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:57:40.330751 21415 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:57:40.331929 21415 hybrid_clock.cc:648] HybridClock initialized: now 1786474660331913 us; error 29 us; skew 500 ppm
Aug 11 18:57:40 dist-test-slave-2j7r krb5kdc[21115](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474660, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:57:40.334712 21415 init.cc:377] Logged in from keytab as kudu/127.27.180.188@KRBTEST.COM (short username kudu)
I20260811 18:57:40.335572 21415 webserver.cc:460] Webserver started at http://127.27.180.188:43859/ using document root <none> and password file <none>
I20260811 18:57:40.336064 21415 fs_manager.cc:359] Metadata directory not provided
I20260811 18:57:40.336126 21415 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:57:40.336305 21415 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:57:40.338266 21415 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474496157106-28370-0/minicluster-data/master-2/data/instance:
uuid: "b3ee966c28ea4700b8caafcbee5e767b"
format_stamp: "Formatted at 2026-08-11 18:57:40 on dist-test-slave-2j7r"
server_key: "e0236e8b6caa4fa474b40d6c92092b44"
server_key_iv: "88f3002451a49a476ddfa9bea86656cd"
server_key_version: "encryptionkey@0"
I20260811 18:57:40.338786 21415 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474496157106-28370-0/minicluster-data/master-2/wal/instance:
uuid: "b3ee966c28ea4700b8caafcbee5e767b"
format_stamp: "Formatted at 2026-08-11 18:57:40 on dist-test-slave-2j7r"
server_key: "e0236e8b6caa4fa474b40d6c92092b44"
server_key_iv: "88f3002451a49a476ddfa9bea86656cd"
server_key_version: "encryptionkey@0"
I20260811 18:57:40.342410 21415 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 18:57:40.344727 21447 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:40.345954 21415 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 18:57:40.346065 21415 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474496157106-28370-0/minicluster-data/master-2/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474496157106-28370-0/minicluster-data/master-2/wal
uuid: "b3ee966c28ea4700b8caafcbee5e767b"
format_stamp: "Formatted at 2026-08-11 18:57:40 on dist-test-slave-2j7r"
server_key: "e0236e8b6caa4fa474b40d6c92092b44"
server_key_iv: "88f3002451a49a476ddfa9bea86656cd"
server_key_version: "encryptionkey@0"
I20260811 18:57:40.346153 21415 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474496157106-28370-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474496157106-28370-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474496157106-28370-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:57:40.363370 21415 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:57:40.366312 21415 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:57:40.366475 21415 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:57:40.373587 21415 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.188:35091
I20260811 18:57:40.373821 21547 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.188:35091 every 8 connection(s)
I20260811 18:57:40.374583 21415 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474496157106-28370-0/minicluster-data/master-2/data/info.pb
I20260811 18:57:40.377264 21550 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:40.381224 21550 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.180.190" port: 32983 } has no permanent_uuid. Determining permanent_uuid...
I20260811 18:57:40.383780 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 21415
I20260811 18:57:40.383913 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786474496157106-28370-0/minicluster-data/master-2/wal/instance
I20260811 18:57:40.384191 28370 external_mini_cluster.cc:1428] Setting key ca0944a14680658e5e9e2746b823016e
Aug 11 18:57:40 dist-test-slave-2j7r krb5kdc[21115](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474660, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.188@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM
Aug 11 18:57:40 dist-test-slave-2j7r krb5kdc[21115](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474659, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.180.188@KRBTEST.COM
I20260811 18:57:40.394582 21274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:40.382973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60722 (local address 127.27.180.190:32983)
0811 18:57:40.383090 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:40.383094 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:40.383846 (+ 752us) server_negotiation.cc:407] Connection header received
0811 18:57:40.384856 (+ 1010us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:40.384860 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:40.384924 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:40.385017 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:40.386303 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:40.386891 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:40.388239 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:40.388572 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:40.388663 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:40.391148 (+ 2485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:40.391152 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:40.391154 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:40.391179 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:40.392474 (+ 1295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:40.393042 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:40.393045 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:40.393047 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:40.393152 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:40.393494 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:40.393497 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:40.393498 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:40.393711 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:40.393814 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:40.394351 (+ 537us) server_negotiation.cc:299] Negotiation successful
0811 18:57:40.394460 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 18:57:40.395453 21552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:40.383277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:32983 (local address 127.0.0.1:60722)
0811 18:57:40.383704 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:40.383745 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:40.384593 (+ 848us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:40.385340 (+ 747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:40.385347 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:40.385687 (+ 340us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:57:40.386151 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:40.386163 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:40.387010 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:40.387013 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:40.388100 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:40.388109 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:40.388702 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:40.388705 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:40.388960 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:40.389499 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:57:40.389514 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:57:40.391003 (+ 1489us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:57:40.392577 (+ 1574us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:40.392587 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:40.392596 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:40.392951 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:40.393240 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:40.393244 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:40.393246 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:40.393406 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:40.393812 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:57:40.393817 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:57:40.394051 (+ 234us) client_negotiation.cc:769] Sending connection context
0811 18:57:40.394236 (+ 185us) client_negotiation.cc:240] Negotiation successful
0811 18:57:40.394505 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"mutex_wait_us":157,"thread_start_us":113,"threads_started":1}
I20260811 18:57:40.395609 21554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:40.384813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55882 (local address 127.27.180.188:35091)
0811 18:57:40.385119 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:40.385130 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:40.385340 (+ 210us) server_negotiation.cc:407] Connection header received
0811 18:57:40.385497 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:40.385504 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:40.385679 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:40.385824 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:40.386575 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:40.387655 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:40.388615 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:40.389167 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:40.389297 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:40.391875 (+ 2578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:40.391882 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:40.391894 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:40.391920 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:40.393593 (+ 1673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:40.394016 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:40.394021 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:40.394022 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:40.394101 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:40.394372 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:40.394374 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:40.394376 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:40.394629 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:40.394806 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:40.395210 (+ 404us) server_negotiation.cc:299] Negotiation successful
0811 18:57:40.395351 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":18,"server-negotiator.queue_time_us":193,"thread_start_us":101,"threads_started":1}
I20260811 18:57:40.396852 21550 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.180.189" port: 32801 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 18:57:40 dist-test-slave-2j7r krb5kdc[21115](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474660, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.188@KRBTEST.COM for kudu/127.27.180.189@KRBTEST.COM
Aug 11 18:57:40 dist-test-slave-2j7r krb5kdc[21115](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474659, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM
I20260811 18:57:40.404230 21552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:40.397476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.189:32801 (local address 127.0.0.1:60144)
0811 18:57:40.397717 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:40.397730 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:40.397808 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:40.398011 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:40.398014 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:40.398157 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:57:40.398236 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:40.398249 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:40.398932 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:40.398935 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:40.399422 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:40.399429 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:40.399841 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:40.399844 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:40.399938 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:40.400323 (+ 385us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:57:40.400336 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:57:40.401447 (+ 1111us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:57:40.402935 (+ 1488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:40.402938 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:40.402940 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:40.403115 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:40.403363 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:40.403365 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:40.403367 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:40.403441 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:40.403779 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:57:40.403782 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:57:40.403868 (+ 86us) client_negotiation.cc:769] Sending connection context
0811 18:57:40.403951 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 18:57:40.404082 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181}
I20260811 18:57:40.404247 21414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:40.397600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60144 (local address 127.27.180.189:32801)
0811 18:57:40.397719 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:40.397724 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:40.397771 (+ 47us) server_negotiation.cc:407] Connection header received
0811 18:57:40.397881 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:40.397884 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:40.397926 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:40.397992 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:40.398355 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:40.398838 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:40.399527 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:40.399749 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:40.399832 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:40.401572 (+ 1740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:40.401575 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:40.401578 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:40.401602 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:40.402837 (+ 1235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:40.403198 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:40.403202 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:40.403204 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:40.403283 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:40.403520 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:40.403523 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:40.403525 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:40.403687 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:40.403778 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:40.404014 (+ 236us) server_negotiation.cc:299] Negotiation successful
0811 18:57:40.404114 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 18:57:40.405261 21550 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.180.188" port: 35091 } has no permanent_uuid. Determining permanent_uuid...
I20260811 18:57:40.406320 21274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:40.397744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60734 (local address 127.27.180.190:32983)
0811 18:57:40.397849 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:40.397854 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:40.397942 (+ 88us) server_negotiation.cc:407] Connection header received
0811 18:57:40.398069 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:40.398073 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:40.398116 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:40.398237 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:40.398823 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:40.399307 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:40.400056 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:40.400284 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:40.400371 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:40.402836 (+ 2465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:40.402840 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:40.402843 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:40.402870 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:40.404201 (+ 1331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:40.404603 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:40.404607 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:40.404609 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:40.404693 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:40.404974 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:40.404977 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:40.404978 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:40.405217 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:40.405454 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:40.406030 (+ 576us) server_negotiation.cc:299] Negotiation successful
0811 18:57:40.406152 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
Aug 11 18:57:40 dist-test-slave-2j7r krb5kdc[21115](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474660, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.188@KRBTEST.COM for kudu/127.27.180.188@KRBTEST.COM
Aug 11 18:57:40 dist-test-slave-2j7r krb5kdc[21115](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474659, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.180.189@KRBTEST.COM
I20260811 18:57:40.412194 21405 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.180.188" port: 35091 } attempt: 3
I20260811 18:57:40.412881 21552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:40.405734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.188:35091 (local address 127.0.0.1:55884)
0811 18:57:40.405999 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:40.406016 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:40.406100 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:40.406332 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:40.406335 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:40.406493 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:57:40.406572 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:40.406578 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:40.407437 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:40.407440 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:40.408078 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:40.408087 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:40.408511 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:40.408515 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:40.408584 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:40.408948 (+ 364us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:57:40.408962 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:57:40.410070 (+ 1108us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:57:40.411481 (+ 1411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:40.411485 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:40.411487 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:40.411714 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:40.411979 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:40.411982 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:40.411984 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:40.412075 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:40.412427 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:57:40.412429 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:57:40.412510 (+ 81us) client_negotiation.cc:769] Sending connection context
0811 18:57:40.412604 (+ 94us) client_negotiation.cc:240] Negotiation successful
0811 18:57:40.412719 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260811 18:57:40.413345 21554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:40.405776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55884 (local address 127.27.180.188:35091)
0811 18:57:40.405867 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:40.405871 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:40.406067 (+ 196us) server_negotiation.cc:407] Connection header received
0811 18:57:40.406182 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:40.406183 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:40.406215 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:40.406298 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:40.406689 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:40.407329 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:40.408179 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:40.408410 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:40.408493 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:40.410214 (+ 1721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:40.410217 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:40.410219 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:40.410240 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:40.411379 (+ 1139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:40.411810 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:40.411814 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:40.411815 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:40.411891 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:40.412151 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:40.412153 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:40.412154 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:40.412312 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:40.412394 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:40.413067 (+ 673us) server_negotiation.cc:299] Negotiation successful
0811 18:57:40.413211 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260811 18:57:40.415550 21414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:40.407377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60148 (local address 127.27.180.189:32801)
0811 18:57:40.407474 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:40.407477 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:40.407572 (+ 95us) server_negotiation.cc:407] Connection header received
0811 18:57:40.407652 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:40.407655 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:40.407695 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:40.407789 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:40.408318 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:40.408779 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:40.409568 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:40.409811 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:40.409923 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:40.412261 (+ 2338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:40.412265 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:40.412267 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:40.412291 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:40.414042 (+ 1751us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:40.414410 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:40.414414 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:40.414416 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:40.414494 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:40.414801 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:40.414805 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:40.414807 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:40.414972 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:40.415182 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:40.415312 (+ 130us) server_negotiation.cc:299] Negotiation successful
0811 18:57:40.415411 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
Aug 11 18:57:40 dist-test-slave-2j7r krb5kdc[21115](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474660, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.189@KRBTEST.COM for kudu/127.27.180.188@KRBTEST.COM
I20260811 18:57:40.417304 21550 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b3ee966c28ea4700b8caafcbee5e767b: Bootstrap starting.
I20260811 18:57:40.417529 21269 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.180.188" port: 35091 } attempt: 3
I20260811 18:57:40.419406 28370 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:57:40.420235 21550 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b3ee966c28ea4700b8caafcbee5e767b: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:40.420887 21412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:40.412627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.188:35091 (local address 127.0.0.1:55886)
0811 18:57:40.412783 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:40.412797 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:40.412863 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:40.413235 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:40.413237 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:40.413405 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:57:40.413511 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:40.413518 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:40.414497 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:40.414500 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:40.415180 (+ 680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:40.415191 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:40.415766 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:40.415770 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:40.415847 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:40.416263 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:57:40.416277 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:57:40.417489 (+ 1212us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:57:40.419377 (+ 1888us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:40.419380 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:40.419383 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:40.419625 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:40.419917 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:40.419920 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:40.419922 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:40.420000 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:40.420450 (+ 450us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:57:40.420453 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:57:40.420553 (+ 100us) client_negotiation.cc:769] Sending connection context
0811 18:57:40.420627 (+ 74us) client_negotiation.cc:240] Negotiation successful
0811 18:57:40.420742 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97}
I20260811 18:57:40.421156 21550 log.cc:824] T 00000000000000000000000000000000 P b3ee966c28ea4700b8caafcbee5e767b: Log is configured to *not* fsync() on all Append() calls
I20260811 18:57:40.421537 21557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:40.412654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55886 (local address 127.27.180.188:35091)
0811 18:57:40.412990 (+ 336us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:40.412999 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:40.413011 (+ 12us) server_negotiation.cc:407] Connection header received
0811 18:57:40.413043 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:40.413046 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:40.413109 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:40.413201 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:40.413616 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:40.414350 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:40.415331 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:40.415641 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:40.416760 (+ 1119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:40.417655 (+ 895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:40.417658 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:40.417661 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:40.417685 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:40.419248 (+ 1563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:40.419740 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:40.419743 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:40.419745 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:40.419825 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:40.420098 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:40.420100 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:40.420102 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:40.420335 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:40.421156 (+ 821us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:40.421228 (+ 72us) server_negotiation.cc:299] Negotiation successful
0811 18:57:40.421351 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":115,"threads_started":1}
I20260811 18:57:40.423394 21550 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b3ee966c28ea4700b8caafcbee5e767b: No bootstrap required, opened a new log
I20260811 18:57:40.424490 21405 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c94f647e51864f6b924f300b69fc79eb: Bootstrap starting.
Aug 11 18:57:40 dist-test-slave-2j7r krb5kdc[21115](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474659, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.190@KRBTEST.COM for kudu/127.27.180.188@KRBTEST.COM
I20260811 18:57:40.427218 21405 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c94f647e51864f6b924f300b69fc79eb: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:40.427527 21550 raft_consensus.cc:348] T 00000000000000000000000000000000 P b3ee966c28ea4700b8caafcbee5e767b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5df196ec71174cdaa2615c8170eda9a5" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 32983 } } peers { permanent_uuid: "c94f647e51864f6b924f300b69fc79eb" member_type: VOTER last_known_addr { host: "127.27.180.189" port: 32801 } } peers { permanent_uuid: "b3ee966c28ea4700b8caafcbee5e767b" member_type: VOTER last_known_addr { host: "127.27.180.188" port: 35091 } }
I20260811 18:57:40.427799 21550 raft_consensus.cc:374] T 00000000000000000000000000000000 P b3ee966c28ea4700b8caafcbee5e767b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:40.427850 21550 raft_consensus.cc:729] T 00000000000000000000000000000000 P b3ee966c28ea4700b8caafcbee5e767b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3ee966c28ea4700b8caafcbee5e767b, State: Initialized, Role: FOLLOWER
I20260811 18:57:40.428146 21405 log.cc:824] T 00000000000000000000000000000000 P c94f647e51864f6b924f300b69fc79eb: Log is configured to *not* fsync() on all Append() calls
I20260811 18:57:40.428488 21550 consensus_queue.cc:260] T 00000000000000000000000000000000 P b3ee966c28ea4700b8caafcbee5e767b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5df196ec71174cdaa2615c8170eda9a5" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 32983 } } peers { permanent_uuid: "c94f647e51864f6b924f300b69fc79eb" member_type: VOTER last_known_addr { host: "127.27.180.189" port: 32801 } } peers { permanent_uuid: "b3ee966c28ea4700b8caafcbee5e767b" member_type: VOTER last_known_addr { host: "127.27.180.188" port: 35091 } }
I20260811 18:57:40.429955 21550 sys_catalog.cc:564] T 00000000000000000000000000000000 P b3ee966c28ea4700b8caafcbee5e767b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:57:40.430536 21405 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c94f647e51864f6b924f300b69fc79eb: No bootstrap required, opened a new log
I20260811 18:57:40.430583 21272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:40.418135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.188:35091 (local address 127.0.0.1:55892)
0811 18:57:40.418346 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:40.418359 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:40.418439 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:40.418858 (+ 419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:40.418862 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:40.419027 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:57:40.419129 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:40.419136 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:40.420156 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:40.420159 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:40.420913 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:40.420925 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:40.421948 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:40.421952 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:40.422042 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:40.422470 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:57:40.422484 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:57:40.426092 (+ 3608us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:57:40.428356 (+ 2264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:40.428359 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:40.428362 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:40.429184 (+ 822us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:40.429530 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:40.429532 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:40.429534 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:40.429626 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:40.430096 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:57:40.430099 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:57:40.430195 (+ 96us) client_negotiation.cc:769] Sending connection context
0811 18:57:40.430288 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 18:57:40.430424 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147}
I20260811 18:57:40.431135 21554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:40.418179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55892 (local address 127.27.180.188:35091)
0811 18:57:40.418280 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:40.418284 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:40.418415 (+ 131us) server_negotiation.cc:407] Connection header received
0811 18:57:40.418660 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:40.418663 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:40.418707 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:40.419240 (+ 533us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:40.419317 (+ 77us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:40.420044 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:40.421316 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:40.421818 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:40.422985 (+ 1167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:40.426301 (+ 3316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:40.426305 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:40.426308 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:40.426333 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:40.428100 (+ 1767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:40.429324 (+ 1224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:40.429328 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:40.429329 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:40.429418 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:40.429735 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:40.429738 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:40.429740 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:40.429959 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:40.430047 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:40.430891 (+ 844us) server_negotiation.cc:299] Negotiation successful
0811 18:57:40.431005 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 18:57:40.432144 21274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:40.421746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60744 (local address 127.27.180.190:32983)
0811 18:57:40.421850 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:40.421854 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:40.423422 (+ 1568us) server_negotiation.cc:407] Connection header received
0811 18:57:40.423583 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:40.423586 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:40.423629 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:40.423701 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:40.424478 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:40.425204 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:40.426344 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:40.426665 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:40.426737 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:40.428298 (+ 1561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:40.428301 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:40.428304 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:40.428329 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:40.429856 (+ 1527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:40.430389 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:40.430392 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:40.430394 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:40.430482 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:40.431067 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:40.431070 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:40.431072 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:40.431307 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:40.431425 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:40.431884 (+ 459us) server_negotiation.cc:299] Negotiation successful
0811 18:57:40.432004 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 18:57:40.435401 21269 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5df196ec71174cdaa2615c8170eda9a5: Bootstrap starting.
I20260811 18:57:40.432657 21579 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3ee966c28ea4700b8caafcbee5e767b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5df196ec71174cdaa2615c8170eda9a5" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 32983 } } peers { permanent_uuid: "c94f647e51864f6b924f300b69fc79eb" member_type: VOTER last_known_addr { host: "127.27.180.189" port: 32801 } } peers { permanent_uuid: "b3ee966c28ea4700b8caafcbee5e767b" member_type: VOTER last_known_addr { host: "127.27.180.188" port: 35091 } } }
I20260811 18:57:40.438019 21269 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5df196ec71174cdaa2615c8170eda9a5: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:40.438926 21269 log.cc:824] T 00000000000000000000000000000000 P 5df196ec71174cdaa2615c8170eda9a5: Log is configured to *not* fsync() on all Append() calls
I20260811 18:57:40.439407 21405 raft_consensus.cc:348] T 00000000000000000000000000000000 P c94f647e51864f6b924f300b69fc79eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5df196ec71174cdaa2615c8170eda9a5" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 32983 } } peers { permanent_uuid: "c94f647e51864f6b924f300b69fc79eb" member_type: VOTER last_known_addr { host: "127.27.180.189" port: 32801 } } peers { permanent_uuid: "b3ee966c28ea4700b8caafcbee5e767b" member_type: VOTER last_known_addr { host: "127.27.180.188" port: 35091 } }
I20260811 18:57:40.439682 21405 raft_consensus.cc:374] T 00000000000000000000000000000000 P c94f647e51864f6b924f300b69fc79eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:40.439769 21405 raft_consensus.cc:729] T 00000000000000000000000000000000 P c94f647e51864f6b924f300b69fc79eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c94f647e51864f6b924f300b69fc79eb, State: Initialized, Role: FOLLOWER
I20260811 18:57:40.440284 21405 consensus_queue.cc:260] T 00000000000000000000000000000000 P c94f647e51864f6b924f300b69fc79eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5df196ec71174cdaa2615c8170eda9a5" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 32983 } } peers { permanent_uuid: "c94f647e51864f6b924f300b69fc79eb" member_type: VOTER last_known_addr { host: "127.27.180.189" port: 32801 } } peers { permanent_uuid: "b3ee966c28ea4700b8caafcbee5e767b" member_type: VOTER last_known_addr { host: "127.27.180.188" port: 35091 } }
I20260811 18:57:40.441182 21269 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5df196ec71174cdaa2615c8170eda9a5: No bootstrap required, opened a new log
I20260811 18:57:40.441586 21579 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3ee966c28ea4700b8caafcbee5e767b [sys.catalog]: This master's current role is: FOLLOWER
W20260811 18:57:40.435328 21595 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b3ee966c28ea4700b8caafcbee5e767b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260811 18:57:40.438455 21415 master_runner.cc:186] Error getting master registration for 127.27.180.190:32983: OK, instance_id { permanent_uuid: "5df196ec71174cdaa2615c8170eda9a5" instance_seqno: 1786474659954209 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 18:57:40.441799 21415 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260811 18:57:40.441859 21595 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:57:40.442054 21405 sys_catalog.cc:564] T 00000000000000000000000000000000 P c94f647e51864f6b924f300b69fc79eb [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:57:40.442972 21557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:40.422807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55908 (local address 127.27.180.188:35091)
0811 18:57:40.428957 (+ 6150us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:40.428962 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:40.433244 (+ 4282us) server_negotiation.cc:407] Connection header received
0811 18:57:40.433500 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:40.433502 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:40.433553 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:40.433603 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:40.434341 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:40.435857 (+ 1516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:40.436750 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:40.437128 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:40.437227 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:40.439239 (+ 2012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:40.439243 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:40.439245 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:40.439270 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:40.440825 (+ 1555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:40.441326 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:40.441330 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:40.441332 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:40.441421 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:40.441730 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:40.441734 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:40.441735 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:40.441951 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:40.442077 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:40.442680 (+ 603us) server_negotiation.cc:299] Negotiation successful
0811 18:57:40.442813 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6092,"thread_start_us":88,"threads_started":1}
I20260811 18:57:40.445240 21269 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5df196ec71174cdaa2615c8170eda9a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5df196ec71174cdaa2615c8170eda9a5" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 32983 } } peers { permanent_uuid: "c94f647e51864f6b924f300b69fc79eb" member_type: VOTER last_known_addr { host: "127.27.180.189" port: 32801 } } peers { permanent_uuid: "b3ee966c28ea4700b8caafcbee5e767b" member_type: VOTER last_known_addr { host: "127.27.180.188" port: 35091 } }
I20260811 18:57:40.445680 21269 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5df196ec71174cdaa2615c8170eda9a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:40.445750 21269 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5df196ec71174cdaa2615c8170eda9a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5df196ec71174cdaa2615c8170eda9a5, State: Initialized, Role: FOLLOWER
I20260811 18:57:40.446236 21269 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5df196ec71174cdaa2615c8170eda9a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5df196ec71174cdaa2615c8170eda9a5" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 32983 } } peers { permanent_uuid: "c94f647e51864f6b924f300b69fc79eb" member_type: VOTER last_known_addr { host: "127.27.180.189" port: 32801 } } peers { permanent_uuid: "b3ee966c28ea4700b8caafcbee5e767b" member_type: VOTER last_known_addr { host: "127.27.180.188" port: 35091 } }
I20260811 18:57:40.447149 21414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:40.425198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60158 (local address 127.27.180.189:32801)
0811 18:57:40.428179 (+ 2981us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:40.428184 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:40.432989 (+ 4805us) server_negotiation.cc:407] Connection header received
0811 18:57:40.434364 (+ 1375us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:40.434367 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:40.434417 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:40.434467 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:40.435535 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:40.436236 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:40.437150 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:40.437443 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:40.437539 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:40.443493 (+ 5954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:40.443498 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:40.443500 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:40.443528 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:40.445071 (+ 1543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:40.445537 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:40.445540 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:40.445542 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:40.445629 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:40.445985 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:40.445988 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:40.445990 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:40.446195 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:40.446314 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:40.446896 (+ 582us) server_negotiation.cc:299] Negotiation successful
0811 18:57:40.447011 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2903}
I20260811 18:57:40.447503 21269 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5df196ec71174cdaa2615c8170eda9a5 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 18:57:40.448621 21621 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c94f647e51864f6b924f300b69fc79eb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:57:40.448691 21621 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:57:40.449141 21599 sys_catalog.cc:455] T 00000000000000000000000000000000 P c94f647e51864f6b924f300b69fc79eb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5df196ec71174cdaa2615c8170eda9a5" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 32983 } } peers { permanent_uuid: "c94f647e51864f6b924f300b69fc79eb" member_type: VOTER last_known_addr { host: "127.27.180.189" port: 32801 } } peers { permanent_uuid: "b3ee966c28ea4700b8caafcbee5e767b" member_type: VOTER last_known_addr { host: "127.27.180.188" port: 35091 } } }
I20260811 18:57:40.449347 21599 sys_catalog.cc:458] T 00000000000000000000000000000000 P c94f647e51864f6b924f300b69fc79eb [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:57:40.446983 21277 master_runner.cc:186] Error getting master registration for 127.27.180.190:32983: OK, instance_id { permanent_uuid: "5df196ec71174cdaa2615c8170eda9a5" instance_seqno: 1786474659954209 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 18:57:40.450500 21277 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260811 18:57:40.450570 21179 tablet.cc:2369] T 00000000000000000000000000000000 P 5df196ec71174cdaa2615c8170eda9a5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 18:57:40.452435 21642 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5df196ec71174cdaa2615c8170eda9a5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:57:40.452517 21642 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:57:40.456146 21135 master_runner.cc:263] No leader master found from master 5df196ec71174cdaa2615c8170eda9a5
I20260811 18:57:40.456220 21135 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 18:57:40.461336 21623 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5df196ec71174cdaa2615c8170eda9a5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5df196ec71174cdaa2615c8170eda9a5" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 32983 } } peers { permanent_uuid: "c94f647e51864f6b924f300b69fc79eb" member_type: VOTER last_known_addr { host: "127.27.180.189" port: 32801 } } peers { permanent_uuid: "b3ee966c28ea4700b8caafcbee5e767b" member_type: VOTER last_known_addr { host: "127.27.180.188" port: 35091 } } }
I20260811 18:57:40.462620 21623 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5df196ec71174cdaa2615c8170eda9a5 [sys.catalog]: This master's current role is: FOLLOWER
W20260811 18:57:40.617777 21460 tablet.cc:2369] T 00000000000000000000000000000000 P b3ee966c28ea4700b8caafcbee5e767b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 18:57:40.655797 21325 tablet.cc:2369] T 00000000000000000000000000000000 P c94f647e51864f6b924f300b69fc79eb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:57:40.811728 21579 raft_consensus.cc:482] T 00000000000000000000000000000000 P b3ee966c28ea4700b8caafcbee5e767b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:40.811951 21579 raft_consensus.cc:504] T 00000000000000000000000000000000 P b3ee966c28ea4700b8caafcbee5e767b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5df196ec71174cdaa2615c8170eda9a5" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 32983 } } peers { permanent_uuid: "c94f647e51864f6b924f300b69fc79eb" member_type: VOTER last_known_addr { host: "127.27.180.189" port: 32801 } } peers { permanent_uuid: "b3ee966c28ea4700b8caafcbee5e767b" member_type: VOTER last_known_addr { host: "127.27.180.188" port: 35091 } }
I20260811 18:57:40.812916 21579 leader_election.cc:288] T 00000000000000000000000000000000 P b3ee966c28ea4700b8caafcbee5e767b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5df196ec71174cdaa2615c8170eda9a5 (127.27.180.190:32983), c94f647e51864f6b924f300b69fc79eb (127.27.180.189:32801)
I20260811 18:57:40.813539 21223 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b3ee966c28ea4700b8caafcbee5e767b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5df196ec71174cdaa2615c8170eda9a5" is_pre_election: true
I20260811 18:57:40.813537 21362 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b3ee966c28ea4700b8caafcbee5e767b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c94f647e51864f6b924f300b69fc79eb" is_pre_election: true
I20260811 18:57:40.813833 21223 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5df196ec71174cdaa2615c8170eda9a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3ee966c28ea4700b8caafcbee5e767b in term 0.
I20260811 18:57:40.813845 21362 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c94f647e51864f6b924f300b69fc79eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3ee966c28ea4700b8caafcbee5e767b in term 0.
I20260811 18:57:40.814263 21453 leader_election.cc:302] T 00000000000000000000000000000000 P b3ee966c28ea4700b8caafcbee5e767b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b3ee966c28ea4700b8caafcbee5e767b, c94f647e51864f6b924f300b69fc79eb; no voters:
I20260811 18:57:40.814536 21579 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b3ee966c28ea4700b8caafcbee5e767b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:40.814627 21579 raft_consensus.cc:482] T 00000000000000000000000000000000 P b3ee966c28ea4700b8caafcbee5e767b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:40.814666 21579 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b3ee966c28ea4700b8caafcbee5e767b [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:40.815825 21579 raft_consensus.cc:504] T 00000000000000000000000000000000 P b3ee966c28ea4700b8caafcbee5e767b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5df196ec71174cdaa2615c8170eda9a5" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 32983 } } peers { permanent_uuid: "c94f647e51864f6b924f300b69fc79eb" member_type: VOTER last_known_addr { host: "127.27.180.189" port: 32801 } } peers { permanent_uuid: "b3ee966c28ea4700b8caafcbee5e767b" member_type: VOTER last_known_addr { host: "127.27.180.188" port: 35091 } }
I20260811 18:57:40.816155 21579 leader_election.cc:288] T 00000000000000000000000000000000 P b3ee966c28ea4700b8caafcbee5e767b [CANDIDATE]: Term 1 election: Requested vote from peers 5df196ec71174cdaa2615c8170eda9a5 (127.27.180.190:32983), c94f647e51864f6b924f300b69fc79eb (127.27.180.189:32801)
I20260811 18:57:40.816542 21223 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b3ee966c28ea4700b8caafcbee5e767b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5df196ec71174cdaa2615c8170eda9a5"
I20260811 18:57:40.816584 21362 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b3ee966c28ea4700b8caafcbee5e767b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c94f647e51864f6b924f300b69fc79eb"
I20260811 18:57:40.816663 21223 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5df196ec71174cdaa2615c8170eda9a5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:40.816715 21362 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c94f647e51864f6b924f300b69fc79eb [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:40.817772 21223 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5df196ec71174cdaa2615c8170eda9a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3ee966c28ea4700b8caafcbee5e767b in term 1.
I20260811 18:57:40.817839 21362 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c94f647e51864f6b924f300b69fc79eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3ee966c28ea4700b8caafcbee5e767b in term 1.
I20260811 18:57:40.818064 21453 leader_election.cc:302] T 00000000000000000000000000000000 P b3ee966c28ea4700b8caafcbee5e767b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5df196ec71174cdaa2615c8170eda9a5, b3ee966c28ea4700b8caafcbee5e767b; no voters:
I20260811 18:57:40.818231 21579 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b3ee966c28ea4700b8caafcbee5e767b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:40.818423 21579 raft_consensus.cc:686] T 00000000000000000000000000000000 P b3ee966c28ea4700b8caafcbee5e767b [term 1 LEADER]: Becoming Leader. State: Replica: b3ee966c28ea4700b8caafcbee5e767b, State: Running, Role: LEADER
I20260811 18:57:40.818729 21579 consensus_queue.cc:237] T 00000000000000000000000000000000 P b3ee966c28ea4700b8caafcbee5e767b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5df196ec71174cdaa2615c8170eda9a5" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 32983 } } peers { permanent_uuid: "c94f647e51864f6b924f300b69fc79eb" member_type: VOTER last_known_addr { host: "127.27.180.189" port: 32801 } } peers { permanent_uuid: "b3ee966c28ea4700b8caafcbee5e767b" member_type: VOTER last_known_addr { host: "127.27.180.188" port: 35091 } }
I20260811 18:57:40.820614 21646 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3ee966c28ea4700b8caafcbee5e767b [sys.catalog]: SysCatalogTable state changed. Reason: New leader b3ee966c28ea4700b8caafcbee5e767b. Latest consensus state: current_term: 1 leader_uuid: "b3ee966c28ea4700b8caafcbee5e767b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5df196ec71174cdaa2615c8170eda9a5" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 32983 } } peers { permanent_uuid: "c94f647e51864f6b924f300b69fc79eb" member_type: VOTER last_known_addr { host: "127.27.180.189" port: 32801 } } peers { permanent_uuid: "b3ee966c28ea4700b8caafcbee5e767b" member_type: VOTER last_known_addr { host: "127.27.180.188" port: 35091 } } }
I20260811 18:57:40.820814 21646 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3ee966c28ea4700b8caafcbee5e767b [sys.catalog]: This master's current role is: LEADER
I20260811 18:57:40.821244 21655 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:57:40.822362 21655 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:57:40.826668 21223 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5df196ec71174cdaa2615c8170eda9a5 [term 1 FOLLOWER]: Refusing update from remote peer b3ee966c28ea4700b8caafcbee5e767b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:57:40.826742 21362 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c94f647e51864f6b924f300b69fc79eb [term 1 FOLLOWER]: Refusing update from remote peer b3ee966c28ea4700b8caafcbee5e767b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:57:40.827230 21579 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b3ee966c28ea4700b8caafcbee5e767b [LEADER]: Connected to new peer: Peer: permanent_uuid: "5df196ec71174cdaa2615c8170eda9a5" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 32983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:40.827440 21646 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b3ee966c28ea4700b8caafcbee5e767b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c94f647e51864f6b924f300b69fc79eb" member_type: VOTER last_known_addr { host: "127.27.180.189" port: 32801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:40.831408 21623 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5df196ec71174cdaa2615c8170eda9a5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b3ee966c28ea4700b8caafcbee5e767b. Latest consensus state: current_term: 1 leader_uuid: "b3ee966c28ea4700b8caafcbee5e767b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5df196ec71174cdaa2615c8170eda9a5" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 32983 } } peers { permanent_uuid: "c94f647e51864f6b924f300b69fc79eb" member_type: VOTER last_known_addr { host: "127.27.180.189" port: 32801 } } peers { permanent_uuid: "b3ee966c28ea4700b8caafcbee5e767b" member_type: VOTER last_known_addr { host: "127.27.180.188" port: 35091 } } }
I20260811 18:57:40.831523 21623 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5df196ec71174cdaa2615c8170eda9a5 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:57:40.831453 21599 sys_catalog.cc:455] T 00000000000000000000000000000000 P c94f647e51864f6b924f300b69fc79eb [sys.catalog]: SysCatalogTable state changed. Reason: New leader b3ee966c28ea4700b8caafcbee5e767b. Latest consensus state: current_term: 1 leader_uuid: "b3ee966c28ea4700b8caafcbee5e767b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5df196ec71174cdaa2615c8170eda9a5" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 32983 } } peers { permanent_uuid: "c94f647e51864f6b924f300b69fc79eb" member_type: VOTER last_known_addr { host: "127.27.180.189" port: 32801 } } peers { permanent_uuid: "b3ee966c28ea4700b8caafcbee5e767b" member_type: VOTER last_known_addr { host: "127.27.180.188" port: 35091 } } }
I20260811 18:57:40.831559 21599 sys_catalog.cc:458] T 00000000000000000000000000000000 P c94f647e51864f6b924f300b69fc79eb [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:57:40.833434 21646 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3ee966c28ea4700b8caafcbee5e767b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b3ee966c28ea4700b8caafcbee5e767b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5df196ec71174cdaa2615c8170eda9a5" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 32983 } } peers { permanent_uuid: "c94f647e51864f6b924f300b69fc79eb" member_type: VOTER last_known_addr { host: "127.27.180.189" port: 32801 } } peers { permanent_uuid: "b3ee966c28ea4700b8caafcbee5e767b" member_type: VOTER last_known_addr { host: "127.27.180.188" port: 35091 } } }
I20260811 18:57:40.833541 21646 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3ee966c28ea4700b8caafcbee5e767b [sys.catalog]: This master's current role is: LEADER
I20260811 18:57:40.833643 21646 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3ee966c28ea4700b8caafcbee5e767b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b3ee966c28ea4700b8caafcbee5e767b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5df196ec71174cdaa2615c8170eda9a5" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 32983 } } peers { permanent_uuid: "c94f647e51864f6b924f300b69fc79eb" member_type: VOTER last_known_addr { host: "127.27.180.189" port: 32801 } } peers { permanent_uuid: "b3ee966c28ea4700b8caafcbee5e767b" member_type: VOTER last_known_addr { host: "127.27.180.188" port: 35091 } } }
I20260811 18:57:40.833707 21646 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3ee966c28ea4700b8caafcbee5e767b [sys.catalog]: This master's current role is: LEADER
I20260811 18:57:40.833825 21599 sys_catalog.cc:455] T 00000000000000000000000000000000 P c94f647e51864f6b924f300b69fc79eb [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b3ee966c28ea4700b8caafcbee5e767b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5df196ec71174cdaa2615c8170eda9a5" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 32983 } } peers { permanent_uuid: "c94f647e51864f6b924f300b69fc79eb" member_type: VOTER last_known_addr { host: "127.27.180.189" port: 32801 } } peers { permanent_uuid: "b3ee966c28ea4700b8caafcbee5e767b" member_type: VOTER last_known_addr { host: "127.27.180.188" port: 35091 } } }
I20260811 18:57:40.833930 21599 sys_catalog.cc:458] T 00000000000000000000000000000000 P c94f647e51864f6b924f300b69fc79eb [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:57:40.834090 21623 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5df196ec71174cdaa2615c8170eda9a5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b3ee966c28ea4700b8caafcbee5e767b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5df196ec71174cdaa2615c8170eda9a5" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 32983 } } peers { permanent_uuid: "c94f647e51864f6b924f300b69fc79eb" member_type: VOTER last_known_addr { host: "127.27.180.189" port: 32801 } } peers { permanent_uuid: "b3ee966c28ea4700b8caafcbee5e767b" member_type: VOTER last_known_addr { host: "127.27.180.188" port: 35091 } } }
I20260811 18:57:40.834199 21623 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5df196ec71174cdaa2615c8170eda9a5 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:57:40.835537 21655 catalog_manager.cc:1347] Generated new cluster ID: 99ea9d762a7f4ad9b25c5b5f49ed9bb7
I20260811 18:57:40.835593 21655 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:57:40.846313 21655 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:57:40.847165 21655 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:57:40.864813 21655 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b3ee966c28ea4700b8caafcbee5e767b: Generated new TSK 0
I20260811 18:57:40.865712 21655 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:57:41.450604 21621 catalog_manager.cc:1259] Loaded cluster ID: 99ea9d762a7f4ad9b25c5b5f49ed9bb7
I20260811 18:57:41.450691 21621 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c94f647e51864f6b924f300b69fc79eb: loading cluster ID for follower catalog manager: success
I20260811 18:57:41.452448 21621 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c94f647e51864f6b924f300b69fc79eb: acquiring CA information for follower catalog manager: success
I20260811 18:57:41.453656 21642 catalog_manager.cc:1259] Loaded cluster ID: 99ea9d762a7f4ad9b25c5b5f49ed9bb7
I20260811 18:57:41.453673 21621 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c94f647e51864f6b924f300b69fc79eb: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 18:57:41.453716 21642 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 5df196ec71174cdaa2615c8170eda9a5: loading cluster ID for follower catalog manager: success
I20260811 18:57:41.455173 21642 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 5df196ec71174cdaa2615c8170eda9a5: acquiring CA information for follower catalog manager: success
I20260811 18:57:41.456162 21642 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 5df196ec71174cdaa2615c8170eda9a5: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 18:57:41.930296 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 21135
I20260811 18:57:41.946614 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 21277
I20260811 18:57:41.958030 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 21415
2026-08-11T18:57:41Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3521 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474496157106-28370-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:57:42 dist-test-slave-2j7r krb5kdc[21680](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:57:42 dist-test-slave-2j7r krb5kdc[21680](info): set up 2 sockets
Aug 11 18:57:42 dist-test-slave-2j7r krb5kdc[21680](info): commencing operation
krb5kdc: starting...
W20260811 18:57:43.185148 28370 mini_kdc.cc:121] Time spent starting KDC: real 1.200s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474496157106-28370-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474496157106-28370-0/krb5kdc/test-user.keytab.
Aug 11 18:57:43 dist-test-slave-2j7r krb5kdc[21680](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474663, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:57:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:57:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.180.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.180.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.180.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab.
Entry for principal kudu/127.27.180.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab.
I20260811 18:57:43.302110 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474496157106-28370-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474496157106-28370-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474496157106-28370-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.190:45035
--webserver_interface=127.27.180.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab
--principal=kudu/127.27.180.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:45795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.180.190:45035
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:57:43.408270 21700 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:57:43.408489 21700 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:57:43.408525 21700 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:57:43.412333 21700 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:57:43.412374 21700 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:57:43.412390 21700 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:57:43.412405 21700 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:57:43.412417 21700 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:57:43.417348 21700 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:45795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474496157106-28370-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474496157106-28370-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.180.190:45035
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab
--principal=kudu/127.27.180.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.190:45035
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.180.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21700
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474496157106-28370-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:57:43.418167 21700 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:57:43.419034 21700 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:57:43.426872 21714 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:43.426892 21711 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:43.426900 21706 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:57:43.427279 21700 server_base.cc:1034] running on GCE node
I20260811 18:57:43.427829 21700 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:57:43.428867 21700 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:57:43.430047 21700 hybrid_clock.cc:648] HybridClock initialized: now 1786474663430031 us; error 57 us; skew 500 ppm
Aug 11 18:57:43 dist-test-slave-2j7r krb5kdc[21680](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474663, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:57:43.436568 21700 init.cc:377] Logged in from keytab as kudu/127.27.180.190@KRBTEST.COM (short username kudu)
I20260811 18:57:43.437474 21700 webserver.cc:460] Webserver started at http://127.27.180.190:43621/ using document root <none> and password file <none>
I20260811 18:57:43.438011 21700 fs_manager.cc:359] Metadata directory not provided
I20260811 18:57:43.438079 21700 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:57:43.438284 21700 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:57:43.440261 21700 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474496157106-28370-0/minicluster-data/master-0/data/instance:
uuid: "c7d36ff97cf04afcae0d8011bef4ebc2"
format_stamp: "Formatted at 2026-08-11 18:57:43 on dist-test-slave-2j7r"
server_key: "03ee51d01e1594889760ab44313564fd"
server_key_iv: "a92d3bcf5155598a1d6114a5d04840ca"
server_key_version: "encryptionkey@0"
I20260811 18:57:43.440815 21700 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474496157106-28370-0/minicluster-data/master-0/wal/instance:
uuid: "c7d36ff97cf04afcae0d8011bef4ebc2"
format_stamp: "Formatted at 2026-08-11 18:57:43 on dist-test-slave-2j7r"
server_key: "03ee51d01e1594889760ab44313564fd"
server_key_iv: "a92d3bcf5155598a1d6114a5d04840ca"
server_key_version: "encryptionkey@0"
I20260811 18:57:43.444361 21700 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260811 18:57:43.446674 21725 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:43.447897 21700 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 18:57:43.448007 21700 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474496157106-28370-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474496157106-28370-0/minicluster-data/master-0/wal
uuid: "c7d36ff97cf04afcae0d8011bef4ebc2"
format_stamp: "Formatted at 2026-08-11 18:57:43 on dist-test-slave-2j7r"
server_key: "03ee51d01e1594889760ab44313564fd"
server_key_iv: "a92d3bcf5155598a1d6114a5d04840ca"
server_key_version: "encryptionkey@0"
I20260811 18:57:43.448102 21700 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474496157106-28370-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474496157106-28370-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474496157106-28370-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:57:43.467918 21700 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:57:43.470569 21700 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:57:43.470753 21700 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:57:43.477792 21700 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.190:45035
I20260811 18:57:43.477818 21815 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.190:45035 every 8 connection(s)
I20260811 18:57:43.478724 21700 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
I20260811 18:57:43.481547 21817 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:43.487188 21817 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7d36ff97cf04afcae0d8011bef4ebc2: Bootstrap starting.
I20260811 18:57:43.487645 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 21700
I20260811 18:57:43.487751 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786474496157106-28370-0/minicluster-data/master-0/wal/instance
I20260811 18:57:43.488008 28370 external_mini_cluster.cc:1428] Setting key 29c47bfa343fbea2bd4a816e1b1f4ed7
I20260811 18:57:43.489786 21817 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c7d36ff97cf04afcae0d8011bef4ebc2: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:43.490522 21817 log.cc:824] T 00000000000000000000000000000000 P c7d36ff97cf04afcae0d8011bef4ebc2: Log is configured to *not* fsync() on all Append() calls
I20260811 18:57:43.492661 21817 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7d36ff97cf04afcae0d8011bef4ebc2: No bootstrap required, opened a new log
I20260811 18:57:43.496565 21817 raft_consensus.cc:348] T 00000000000000000000000000000000 P c7d36ff97cf04afcae0d8011bef4ebc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7d36ff97cf04afcae0d8011bef4ebc2" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 45035 } }
I20260811 18:57:43.496779 21817 raft_consensus.cc:374] T 00000000000000000000000000000000 P c7d36ff97cf04afcae0d8011bef4ebc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:43.496836 21817 raft_consensus.cc:729] T 00000000000000000000000000000000 P c7d36ff97cf04afcae0d8011bef4ebc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7d36ff97cf04afcae0d8011bef4ebc2, State: Initialized, Role: FOLLOWER
I20260811 18:57:43.497287 21817 consensus_queue.cc:260] T 00000000000000000000000000000000 P c7d36ff97cf04afcae0d8011bef4ebc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7d36ff97cf04afcae0d8011bef4ebc2" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 45035 } }
I20260811 18:57:43.497427 21817 raft_consensus.cc:388] T 00000000000000000000000000000000 P c7d36ff97cf04afcae0d8011bef4ebc2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:57:43.497493 21817 raft_consensus.cc:482] T 00000000000000000000000000000000 P c7d36ff97cf04afcae0d8011bef4ebc2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:57:43.497591 21817 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c7d36ff97cf04afcae0d8011bef4ebc2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:43.498414 21817 raft_consensus.cc:504] T 00000000000000000000000000000000 P c7d36ff97cf04afcae0d8011bef4ebc2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7d36ff97cf04afcae0d8011bef4ebc2" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 45035 } }
I20260811 18:57:43.498724 21817 leader_election.cc:302] T 00000000000000000000000000000000 P c7d36ff97cf04afcae0d8011bef4ebc2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c7d36ff97cf04afcae0d8011bef4ebc2; no voters:
I20260811 18:57:43.499047 21817 leader_election.cc:288] T 00000000000000000000000000000000 P c7d36ff97cf04afcae0d8011bef4ebc2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:57:43.499119 21825 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c7d36ff97cf04afcae0d8011bef4ebc2 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:43.499347 21825 raft_consensus.cc:686] T 00000000000000000000000000000000 P c7d36ff97cf04afcae0d8011bef4ebc2 [term 1 LEADER]: Becoming Leader. State: Replica: c7d36ff97cf04afcae0d8011bef4ebc2, State: Running, Role: LEADER
I20260811 18:57:43.499693 21825 consensus_queue.cc:237] T 00000000000000000000000000000000 P c7d36ff97cf04afcae0d8011bef4ebc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7d36ff97cf04afcae0d8011bef4ebc2" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 45035 } }
I20260811 18:57:43.500332 21817 sys_catalog.cc:564] T 00000000000000000000000000000000 P c7d36ff97cf04afcae0d8011bef4ebc2 [sys.catalog]: configured and running, proceeding with master startup.
Aug 11 18:57:43 dist-test-slave-2j7r krb5kdc[21680](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474663, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM
I20260811 18:57:43.501385 21828 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7d36ff97cf04afcae0d8011bef4ebc2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c7d36ff97cf04afcae0d8011bef4ebc2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7d36ff97cf04afcae0d8011bef4ebc2" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 45035 } } }
I20260811 18:57:43.501506 21828 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7d36ff97cf04afcae0d8011bef4ebc2 [sys.catalog]: This master's current role is: LEADER
I20260811 18:57:43.501595 21830 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7d36ff97cf04afcae0d8011bef4ebc2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c7d36ff97cf04afcae0d8011bef4ebc2. Latest consensus state: current_term: 1 leader_uuid: "c7d36ff97cf04afcae0d8011bef4ebc2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7d36ff97cf04afcae0d8011bef4ebc2" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 45035 } } }
I20260811 18:57:43.501668 21830 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7d36ff97cf04afcae0d8011bef4ebc2 [sys.catalog]: This master's current role is: LEADER
I20260811 18:57:43.502004 21839 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:57:43.505262 21839 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:57:43.507324 21821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:43.489243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35076 (local address 127.27.180.190:45035)
0811 18:57:43.489633 (+ 390us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:43.489648 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:43.489685 (+ 37us) server_negotiation.cc:407] Connection header received
0811 18:57:43.490327 (+ 642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:43.490344 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:43.490665 (+ 321us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:43.490998 (+ 333us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:43.491773 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:43.492774 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:43.493939 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:43.494394 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:43.494505 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:43.501853 (+ 7348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:43.501860 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:43.501877 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:43.501910 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:43.503966 (+ 2056us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:43.504505 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:43.504511 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:43.504517 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:43.504656 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:43.504964 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:43.504968 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:43.504969 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:43.505493 (+ 524us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:43.505702 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:43.506049 (+ 347us) server_negotiation.cc:299] Negotiation successful
0811 18:57:43.506372 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":253,"thread_start_us":108,"threads_started":1}
I20260811 18:57:43.510903 28370 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:57:43.511216 21839 catalog_manager.cc:1347] Generated new cluster ID: d545ea505a5b4c8385ae97efddb782ac
I20260811 18:57:43.511278 21839 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:57:43.518424 21839 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:57:43.519191 21839 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:57:43.529328 21839 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c7d36ff97cf04afcae0d8011bef4ebc2: Generated new TSK 0
I20260811 18:57:43.529928 21839 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:57:43.566502 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 21700
2026-08-11T18:57:43Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1616 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:57:43 dist-test-slave-2j7r krb5kdc[21860](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:57:43 dist-test-slave-2j7r krb5kdc[21860](info): set up 2 sockets
Aug 11 18:57:43 dist-test-slave-2j7r krb5kdc[21860](info): commencing operation
krb5kdc: starting...
W20260811 18:57:44.783380 28370 mini_kdc.cc:121] Time spent starting KDC: real 1.180s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/krb5kdc/test-user.keytab.
Aug 11 18:57:44 dist-test-slave-2j7r krb5kdc[21860](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474664, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:57:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:57:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.180.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.180.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.180.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab.
Entry for principal kudu/127.27.180.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab.
I20260811 18:57:44.903643 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.190:40989
--webserver_interface=127.27.180.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab
--principal=kudu/127.27.180.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:42327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.180.190:40989
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:57:45.015776 21879 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:57:45.016021 21879 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:57:45.016076 21879 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:57:45.020046 21879 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:57:45.020094 21879 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:57:45.020120 21879 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:57:45.020143 21879 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:57:45.020171 21879 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:57:45.025245 21879 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:42327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.180.190:40989
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab
--principal=kudu/127.27.180.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.190:40989
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.180.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21879
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:57:45.026237 21879 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:57:45.027264 21879 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:57:45.035276 21889 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:45.035329 21892 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:45.035358 21888 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:57:45.035583 21879 server_base.cc:1034] running on GCE node
I20260811 18:57:45.036098 21879 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:57:45.037142 21879 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:57:45.038322 21879 hybrid_clock.cc:648] HybridClock initialized: now 1786474665038301 us; error 38 us; skew 500 ppm
Aug 11 18:57:45 dist-test-slave-2j7r krb5kdc[21860](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474665, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:57:45.041031 21879 init.cc:377] Logged in from keytab as kudu/127.27.180.190@KRBTEST.COM (short username kudu)
I20260811 18:57:45.041878 21879 webserver.cc:460] Webserver started at http://127.27.180.190:37621/ using document root <none> and password file <none>
I20260811 18:57:45.042366 21879 fs_manager.cc:359] Metadata directory not provided
I20260811 18:57:45.042424 21879 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:57:45.042586 21879 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:57:45.044480 21879 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/master-0/data/instance:
uuid: "46e6feef897141feb6bf6aba7a68a7ce"
format_stamp: "Formatted at 2026-08-11 18:57:45 on dist-test-slave-2j7r"
server_key: "446f7ff2aba1006e44a2e51af698b0e1"
server_key_iv: "409541b09efda84ff9e45bf5477e5af7"
server_key_version: "encryptionkey@0"
I20260811 18:57:45.044983 21879 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/master-0/wal/instance:
uuid: "46e6feef897141feb6bf6aba7a68a7ce"
format_stamp: "Formatted at 2026-08-11 18:57:45 on dist-test-slave-2j7r"
server_key: "446f7ff2aba1006e44a2e51af698b0e1"
server_key_iv: "409541b09efda84ff9e45bf5477e5af7"
server_key_version: "encryptionkey@0"
I20260811 18:57:45.048476 21879 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.002s
I20260811 18:57:45.050736 21905 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:45.051867 21879 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 18:57:45.051975 21879 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/master-0/wal
uuid: "46e6feef897141feb6bf6aba7a68a7ce"
format_stamp: "Formatted at 2026-08-11 18:57:45 on dist-test-slave-2j7r"
server_key: "446f7ff2aba1006e44a2e51af698b0e1"
server_key_iv: "409541b09efda84ff9e45bf5477e5af7"
server_key_version: "encryptionkey@0"
I20260811 18:57:45.052062 21879 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:57:45.068233 21879 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:57:45.071218 21879 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:57:45.071400 21879 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:57:45.078262 21879 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.190:40989
I20260811 18:57:45.078541 22001 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.190:40989 every 8 connection(s)
I20260811 18:57:45.079195 21879 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
I20260811 18:57:45.079726 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 21879
I20260811 18:57:45.079861 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/master-0/wal/instance
I20260811 18:57:45.080124 28370 external_mini_cluster.cc:1428] Setting key 6e4555d8818b2a446e88cf30dcb29acb
I20260811 18:57:45.082173 22002 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:45.088511 22002 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46e6feef897141feb6bf6aba7a68a7ce: Bootstrap starting.
Aug 11 18:57:45 dist-test-slave-2j7r krb5kdc[21860](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474664, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM
I20260811 18:57:45.091060 22002 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 46e6feef897141feb6bf6aba7a68a7ce: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:45.091919 22002 log.cc:824] T 00000000000000000000000000000000 P 46e6feef897141feb6bf6aba7a68a7ce: Log is configured to *not* fsync() on all Append() calls
I20260811 18:57:45.093730 22002 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46e6feef897141feb6bf6aba7a68a7ce: No bootstrap required, opened a new log
I20260811 18:57:45.094357 22005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:45.081469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58902 (local address 127.27.180.190:40989)
0811 18:57:45.081896 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:45.081916 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:45.081957 (+ 41us) server_negotiation.cc:407] Connection header received
0811 18:57:45.082756 (+ 799us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:45.082790 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:45.083182 (+ 392us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:45.083522 (+ 340us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:45.084173 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:45.085701 (+ 1528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:45.086575 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:45.087248 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:45.087402 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:45.089888 (+ 2486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:45.089894 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:45.089908 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:45.089936 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:45.091713 (+ 1777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:45.092127 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:45.092132 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:45.092136 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:45.092249 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:45.092501 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:45.092505 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:45.092506 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:45.092878 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:45.093052 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:45.093370 (+ 318us) server_negotiation.cc:299] Negotiation successful
0811 18:57:45.093636 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":118,"threads_started":1}
I20260811 18:57:45.097215 22002 raft_consensus.cc:348] T 00000000000000000000000000000000 P 46e6feef897141feb6bf6aba7a68a7ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46e6feef897141feb6bf6aba7a68a7ce" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 40989 } }
I20260811 18:57:45.097429 22002 raft_consensus.cc:374] T 00000000000000000000000000000000 P 46e6feef897141feb6bf6aba7a68a7ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:45.097486 22002 raft_consensus.cc:729] T 00000000000000000000000000000000 P 46e6feef897141feb6bf6aba7a68a7ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46e6feef897141feb6bf6aba7a68a7ce, State: Initialized, Role: FOLLOWER
I20260811 18:57:45.097886 22002 consensus_queue.cc:260] T 00000000000000000000000000000000 P 46e6feef897141feb6bf6aba7a68a7ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46e6feef897141feb6bf6aba7a68a7ce" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 40989 } }
I20260811 18:57:45.098019 22002 raft_consensus.cc:388] T 00000000000000000000000000000000 P 46e6feef897141feb6bf6aba7a68a7ce [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:57:45.098083 22002 raft_consensus.cc:482] T 00000000000000000000000000000000 P 46e6feef897141feb6bf6aba7a68a7ce [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:57:45.098177 22002 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 46e6feef897141feb6bf6aba7a68a7ce [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:45.099009 22002 raft_consensus.cc:504] T 00000000000000000000000000000000 P 46e6feef897141feb6bf6aba7a68a7ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46e6feef897141feb6bf6aba7a68a7ce" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 40989 } }
I20260811 18:57:45.099321 22002 leader_election.cc:302] T 00000000000000000000000000000000 P 46e6feef897141feb6bf6aba7a68a7ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 46e6feef897141feb6bf6aba7a68a7ce; no voters:
I20260811 18:57:45.099623 22002 leader_election.cc:288] T 00000000000000000000000000000000 P 46e6feef897141feb6bf6aba7a68a7ce [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:57:45.099717 22010 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 46e6feef897141feb6bf6aba7a68a7ce [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:45.099936 22010 raft_consensus.cc:686] T 00000000000000000000000000000000 P 46e6feef897141feb6bf6aba7a68a7ce [term 1 LEADER]: Becoming Leader. State: Replica: 46e6feef897141feb6bf6aba7a68a7ce, State: Running, Role: LEADER
I20260811 18:57:45.100265 22010 consensus_queue.cc:237] T 00000000000000000000000000000000 P 46e6feef897141feb6bf6aba7a68a7ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46e6feef897141feb6bf6aba7a68a7ce" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 40989 } }
I20260811 18:57:45.100797 22002 sys_catalog.cc:564] T 00000000000000000000000000000000 P 46e6feef897141feb6bf6aba7a68a7ce [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:57:45.101804 22012 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46e6feef897141feb6bf6aba7a68a7ce [sys.catalog]: SysCatalogTable state changed. Reason: New leader 46e6feef897141feb6bf6aba7a68a7ce. Latest consensus state: current_term: 1 leader_uuid: "46e6feef897141feb6bf6aba7a68a7ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46e6feef897141feb6bf6aba7a68a7ce" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 40989 } } }
I20260811 18:57:45.101933 22012 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46e6feef897141feb6bf6aba7a68a7ce [sys.catalog]: This master's current role is: LEADER
I20260811 18:57:45.102202 22011 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46e6feef897141feb6bf6aba7a68a7ce [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "46e6feef897141feb6bf6aba7a68a7ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46e6feef897141feb6bf6aba7a68a7ce" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 40989 } } }
I20260811 18:57:45.102279 22011 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46e6feef897141feb6bf6aba7a68a7ce [sys.catalog]: This master's current role is: LEADER
I20260811 18:57:45.102313 22026 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:57:45.105329 22026 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:57:45.110839 22026 catalog_manager.cc:1347] Generated new cluster ID: fe5b4857e0284e22ba246b0656cb5426
I20260811 18:57:45.110903 22026 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:57:45.120400 22026 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:57:45.121585 22026 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:57:45.141109 22026 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 46e6feef897141feb6bf6aba7a68a7ce: Generated new TSK 0
I20260811 18:57:45.141706 22026 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.180.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.180.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.180.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.129.keytab.
Entry for principal kudu/127.27.180.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.129.keytab.
I20260811 18:57:45.173302 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.180.129:0
--local_ip_for_outbound_sockets=127.27.180.129
--webserver_interface=127.27.180.129
--webserver_port=0
--tserver_master_addrs=127.27.180.190:40989
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.129.keytab
--principal=kudu/127.27.180.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:42327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:57:45.283633 22049 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:57:45.283861 22049 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:57:45.283901 22049 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:57:45.287788 22049 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:57:45.287837 22049 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:57:45.287895 22049 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.180.129
I20260811 18:57:45.292847 22049 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:42327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.129.keytab
--principal=kudu/127.27.180.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.180.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.180.129
--webserver_port=0
--tserver_master_addrs=127.27.180.190:40989
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22049
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.180.129
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:57:45.293789 22049 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:57:45.294714 22049 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:57:45.303967 22058 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:45.303977 22060 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:57:45.304127 22049 server_base.cc:1034] running on GCE node
W20260811 18:57:45.304039 22067 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:57:45.304593 22049 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:57:45.305215 22049 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:57:45.306396 22049 hybrid_clock.cc:648] HybridClock initialized: now 1786474665306381 us; error 28 us; skew 500 ppm
Aug 11 18:57:45 dist-test-slave-2j7r krb5kdc[21860](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474665, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:57:45.308972 22049 init.cc:377] Logged in from keytab as kudu/127.27.180.129@KRBTEST.COM (short username kudu)
I20260811 18:57:45.309818 22049 webserver.cc:460] Webserver started at http://127.27.180.129:46263/ using document root <none> and password file <none>
I20260811 18:57:45.310328 22049 fs_manager.cc:359] Metadata directory not provided
I20260811 18:57:45.310393 22049 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:57:45.310595 22049 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:57:45.312497 22049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-0/data/instance:
uuid: "caf849fdfdc048128ec5b5502845312a"
format_stamp: "Formatted at 2026-08-11 18:57:45 on dist-test-slave-2j7r"
server_key: "9bf6192df9a7f04098127322b8a2ef92"
server_key_iv: "bc7f2259ae066343b8817f074423818a"
server_key_version: "encryptionkey@0"
I20260811 18:57:45.313019 22049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-0/wal/instance:
uuid: "caf849fdfdc048128ec5b5502845312a"
format_stamp: "Formatted at 2026-08-11 18:57:45 on dist-test-slave-2j7r"
server_key: "9bf6192df9a7f04098127322b8a2ef92"
server_key_iv: "bc7f2259ae066343b8817f074423818a"
server_key_version: "encryptionkey@0"
I20260811 18:57:45.316537 22049 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260811 18:57:45.318903 22075 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:45.320003 22049 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 18:57:45.320108 22049 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-0/wal
uuid: "caf849fdfdc048128ec5b5502845312a"
format_stamp: "Formatted at 2026-08-11 18:57:45 on dist-test-slave-2j7r"
server_key: "9bf6192df9a7f04098127322b8a2ef92"
server_key_iv: "bc7f2259ae066343b8817f074423818a"
server_key_version: "encryptionkey@0"
I20260811 18:57:45.320199 22049 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:57:45.343497 22049 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:57:45.346156 22049 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:57:45.346323 22049 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:57:45.346875 22049 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:57:45.347638 22049 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:57:45.347683 22049 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:45.347746 22049 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:57:45.347772 22049 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:45.356020 22049 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.129:41443
I20260811 18:57:45.356086 22285 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.129:41443 every 8 connection(s)
I20260811 18:57:45.356863 22049 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-0/data/info.pb
I20260811 18:57:45.359210 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 22049
I20260811 18:57:45.359333 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-0/wal/instance
I20260811 18:57:45.359601 28370 external_mini_cluster.cc:1428] Setting key b1dc3307d38dda6ab23859089288c5b8
Aug 11 18:57:45 dist-test-slave-2j7r krb5kdc[21860](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474665, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.129@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.27.180.130@KRBTEST.COM; defaulting to no policy
I20260811 18:57:45.370416 22005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:45.358477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.129:58835 (local address 127.27.180.190:40989)
0811 18:57:45.358602 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:45.358605 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:45.359717 (+ 1112us) server_negotiation.cc:407] Connection header received
0811 18:57:45.361179 (+ 1462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:45.361182 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:45.361236 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:45.361310 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:45.362405 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:45.362929 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:45.364032 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:45.364264 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:45.364351 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:45.366927 (+ 2576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:45.366932 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:45.366935 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:45.366965 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:45.368440 (+ 1475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:45.368926 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:45.368930 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:45.368932 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:45.369019 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:45.369392 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:45.369396 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:45.369398 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:45.369615 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:45.369718 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:45.370180 (+ 462us) server_negotiation.cc:299] Negotiation successful
0811 18:57:45.370293 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 18:57:45.371178 22289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:45.358805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:40989 (local address 127.27.180.129:58835)
0811 18:57:45.359567 (+ 762us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:45.359614 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:45.360974 (+ 1360us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:45.361481 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:45.361488 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:45.361820 (+ 332us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:57:45.362270 (+ 450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:45.362283 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:45.363044 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:45.363048 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:45.363917 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:45.363927 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:45.364398 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:45.364402 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:45.364574 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:45.365105 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:57:45.365128 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:57:45.366740 (+ 1612us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:57:45.368569 (+ 1829us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:45.368574 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:45.368585 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:45.368828 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:45.369137 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:45.369140 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:45.369141 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:45.369295 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:45.369734 (+ 439us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:57:45.369741 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:57:45.369970 (+ 229us) client_negotiation.cc:769] Sending connection context
0811 18:57:45.370122 (+ 152us) client_negotiation.cc:240] Negotiation successful
0811 18:57:45.370390 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":570,"thread_start_us":107,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.180.130@KRBTEST.COM" created.
I20260811 18:57:45.372376 22286 heartbeater.cc:347] Connected to a master server at 127.27.180.190:40989
I20260811 18:57:45.372609 22286 heartbeater.cc:464] Registering TS with master...
I20260811 18:57:45.373198 22286 heartbeater.cc:511] Master 127.27.180.190:40989 requested a full tablet report, sending...
I20260811 18:57:45.374752 21937 ts_manager.cc:194] Registered new tserver with Master: caf849fdfdc048128ec5b5502845312a (127.27.180.129:41443)
I20260811 18:57:45.376204 21937 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.180.129@KRBTEST.COM'} at 127.27.180.129:58835
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.180.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.130.keytab.
Entry for principal kudu/127.27.180.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.130.keytab.
I20260811 18:57:45.387638 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.180.130:0
--local_ip_for_outbound_sockets=127.27.180.130
--webserver_interface=127.27.180.130
--webserver_port=0
--tserver_master_addrs=127.27.180.190:40989
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.130.keytab
--principal=kudu/127.27.180.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:42327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:57:45.495299 22295 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:57:45.495529 22295 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:57:45.495569 22295 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:57:45.499747 22295 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:57:45.499794 22295 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:57:45.499847 22295 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.180.130
I20260811 18:57:45.504968 22295 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:42327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.130.keytab
--principal=kudu/127.27.180.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.180.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.180.130
--webserver_port=0
--tserver_master_addrs=127.27.180.190:40989
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22295
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.180.130
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:57:45.506032 22295 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:57:45.506975 22295 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:57:45.516021 22310 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:45.516026 22316 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:45.516088 22313 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:57:45.516350 22295 server_base.cc:1034] running on GCE node
I20260811 18:57:45.516739 22295 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:57:45.517349 22295 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:57:45.518512 22295 hybrid_clock.cc:648] HybridClock initialized: now 1786474665518494 us; error 28 us; skew 500 ppm
Aug 11 18:57:45 dist-test-slave-2j7r krb5kdc[21860](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474665, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:57:45.521077 22295 init.cc:377] Logged in from keytab as kudu/127.27.180.130@KRBTEST.COM (short username kudu)
I20260811 18:57:45.521947 22295 webserver.cc:460] Webserver started at http://127.27.180.130:40669/ using document root <none> and password file <none>
I20260811 18:57:45.522462 22295 fs_manager.cc:359] Metadata directory not provided
I20260811 18:57:45.522528 22295 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:57:45.522720 22295 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:57:45.524670 22295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-1/data/instance:
uuid: "d37ed58d1e4e4273a7ad3a2a20681c90"
format_stamp: "Formatted at 2026-08-11 18:57:45 on dist-test-slave-2j7r"
server_key: "15916bcb143779610fea979bb97c7f57"
server_key_iv: "e234587a7b46cce4a2a22c70a0ec694d"
server_key_version: "encryptionkey@0"
I20260811 18:57:45.525270 22295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-1/wal/instance:
uuid: "d37ed58d1e4e4273a7ad3a2a20681c90"
format_stamp: "Formatted at 2026-08-11 18:57:45 on dist-test-slave-2j7r"
server_key: "15916bcb143779610fea979bb97c7f57"
server_key_iv: "e234587a7b46cce4a2a22c70a0ec694d"
server_key_version: "encryptionkey@0"
I20260811 18:57:45.528657 22295 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:57:45.530891 22327 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:45.532011 22295 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 18:57:45.532116 22295 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-1/wal
uuid: "d37ed58d1e4e4273a7ad3a2a20681c90"
format_stamp: "Formatted at 2026-08-11 18:57:45 on dist-test-slave-2j7r"
server_key: "15916bcb143779610fea979bb97c7f57"
server_key_iv: "e234587a7b46cce4a2a22c70a0ec694d"
server_key_version: "encryptionkey@0"
I20260811 18:57:45.532204 22295 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:57:45.543493 22295 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:57:45.546619 22295 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:57:45.546797 22295 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:57:45.547379 22295 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:57:45.548198 22295 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:57:45.548246 22295 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:45.548300 22295 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:57:45.548328 22295 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:45.556752 22295 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.130:36347
I20260811 18:57:45.556793 22538 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.130:36347 every 8 connection(s)
I20260811 18:57:45.557708 22295 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-1/data/info.pb
I20260811 18:57:45.563633 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 22295
I20260811 18:57:45.563784 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-1/wal/instance
I20260811 18:57:45.564074 28370 external_mini_cluster.cc:1428] Setting key 3fbb41e13e1d534b25c0bdb19356557d
Aug 11 18:57:45 dist-test-slave-2j7r krb5kdc[21860](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474665, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.130@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM
I20260811 18:57:45.571501 22005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:45.559366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.130:60649 (local address 127.27.180.190:40989)
0811 18:57:45.559503 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:45.559507 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:45.560301 (+ 794us) server_negotiation.cc:407] Connection header received
0811 18:57:45.561701 (+ 1400us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:45.561705 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:45.561768 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:45.561843 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:45.563039 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:45.563677 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:45.565011 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:45.565363 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:45.565465 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:45.567877 (+ 2412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:45.567881 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:45.567883 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:45.567907 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:45.569390 (+ 1483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:45.569881 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:45.569884 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:45.569886 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:45.569987 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:45.570308 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:45.570312 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:45.570314 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:45.570574 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:45.570683 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:45.571185 (+ 502us) server_negotiation.cc:299] Negotiation successful
0811 18:57:45.571315 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 18:57:45.572222 22542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:45.559715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:40989 (local address 127.27.180.130:60649)
0811 18:57:45.560156 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:45.560199 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:45.561489 (+ 1290us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:45.562028 (+ 539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:45.562035 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:45.562395 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:57:45.562880 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:45.562894 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:45.563840 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:45.563844 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:45.564901 (+ 1057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:45.564912 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:45.565542 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:45.565547 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:45.565713 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:45.566268 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:57:45.566284 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:57:45.567709 (+ 1425us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:57:45.569512 (+ 1803us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:45.569517 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:45.569530 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:45.569779 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:45.570070 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:45.570073 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:45.570074 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:45.570208 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:45.570691 (+ 483us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:57:45.570697 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:57:45.570939 (+ 242us) client_negotiation.cc:769] Sending connection context
0811 18:57:45.571100 (+ 161us) client_negotiation.cc:240] Negotiation successful
0811 18:57:45.571351 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":117,"threads_started":1}
I20260811 18:57:45.573547 22540 heartbeater.cc:347] Connected to a master server at 127.27.180.190:40989
I20260811 18:57:45.573781 22540 heartbeater.cc:464] Registering TS with master...
I20260811 18:57:45.574261 22540 heartbeater.cc:511] Master 127.27.180.190:40989 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.27.180.131@KRBTEST.COM; defaulting to no policy
I20260811 18:57:45.575310 21937 ts_manager.cc:194] Registered new tserver with Master: d37ed58d1e4e4273a7ad3a2a20681c90 (127.27.180.130:36347)
I20260811 18:57:45.575932 21937 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.180.130@KRBTEST.COM'} at 127.27.180.130:60649
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.180.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.180.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.131.keytab.
Entry for principal kudu/127.27.180.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.131.keytab.
I20260811 18:57:45.591704 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.180.131:0
--local_ip_for_outbound_sockets=127.27.180.131
--webserver_interface=127.27.180.131
--webserver_port=0
--tserver_master_addrs=127.27.180.190:40989
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.131.keytab
--principal=kudu/127.27.180.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:42327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:57:45.702174 22551 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:57:45.702419 22551 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:57:45.702461 22551 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:57:45.706511 22551 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:57:45.706570 22551 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:57:45.706625 22551 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.180.131
I20260811 18:57:45.711764 22551 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:42327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.131.keytab
--principal=kudu/127.27.180.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.180.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.180.131
--webserver_port=0
--tserver_master_addrs=127.27.180.190:40989
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22551
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.180.131
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:57:45.712803 22551 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:57:45.713812 22551 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:57:45.723054 22557 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:45.723043 22561 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:57:45.723158 22551 server_base.cc:1034] running on GCE node
W20260811 18:57:45.723054 22558 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:57:45.723690 22551 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:57:45.724354 22551 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:57:45.725526 22551 hybrid_clock.cc:648] HybridClock initialized: now 1786474665725511 us; error 24 us; skew 500 ppm
Aug 11 18:57:45 dist-test-slave-2j7r krb5kdc[21860](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474665, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:57:45.728111 22551 init.cc:377] Logged in from keytab as kudu/127.27.180.131@KRBTEST.COM (short username kudu)
I20260811 18:57:45.728894 22551 webserver.cc:460] Webserver started at http://127.27.180.131:46285/ using document root <none> and password file <none>
I20260811 18:57:45.729434 22551 fs_manager.cc:359] Metadata directory not provided
I20260811 18:57:45.729492 22551 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:57:45.729660 22551 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:57:45.731551 22551 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-2/data/instance:
uuid: "1efdcbeb44e94a3bb3acb45daa0e7197"
format_stamp: "Formatted at 2026-08-11 18:57:45 on dist-test-slave-2j7r"
server_key: "702b7bf9382ecdd0b10b3c3fd01da645"
server_key_iv: "5326459e8b34be2c10b5b59c40bac440"
server_key_version: "encryptionkey@0"
I20260811 18:57:45.732041 22551 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-2/wal/instance:
uuid: "1efdcbeb44e94a3bb3acb45daa0e7197"
format_stamp: "Formatted at 2026-08-11 18:57:45 on dist-test-slave-2j7r"
server_key: "702b7bf9382ecdd0b10b3c3fd01da645"
server_key_iv: "5326459e8b34be2c10b5b59c40bac440"
server_key_version: "encryptionkey@0"
I20260811 18:57:45.735517 22551 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260811 18:57:45.737915 22577 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:45.739095 22551 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 18:57:45.739189 22551 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-2/wal
uuid: "1efdcbeb44e94a3bb3acb45daa0e7197"
format_stamp: "Formatted at 2026-08-11 18:57:45 on dist-test-slave-2j7r"
server_key: "702b7bf9382ecdd0b10b3c3fd01da645"
server_key_iv: "5326459e8b34be2c10b5b59c40bac440"
server_key_version: "encryptionkey@0"
I20260811 18:57:45.739307 22551 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:57:45.762696 22551 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:57:45.765483 22551 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:57:45.765654 22551 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:57:45.766247 22551 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:57:45.767035 22551 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:57:45.767082 22551 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:45.767120 22551 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:57:45.767148 22551 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:45.775174 22551 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.131:45203
I20260811 18:57:45.775202 22773 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.131:45203 every 8 connection(s)
I20260811 18:57:45.776000 22551 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-2/data/info.pb
I20260811 18:57:45.779960 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 22551
I20260811 18:57:45.780076 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-2/wal/instance
I20260811 18:57:45.780313 28370 external_mini_cluster.cc:1428] Setting key 5a0151d31204e7fa9b211615fa378c6f
Aug 11 18:57:45 dist-test-slave-2j7r krb5kdc[21860](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474665, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.131@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM
I20260811 18:57:45.788398 22005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:45.777673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.131:48465 (local address 127.27.180.190:40989)
0811 18:57:45.777816 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:45.777821 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:45.778631 (+ 810us) server_negotiation.cc:407] Connection header received
0811 18:57:45.779962 (+ 1331us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:45.779966 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:45.780025 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:45.780093 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:45.781450 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:45.782032 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:45.783181 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:45.783408 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:45.783486 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:45.785733 (+ 2247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:45.785736 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:45.785738 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:45.785764 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:45.786850 (+ 1086us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:45.787271 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:45.787273 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:45.787274 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:45.787341 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:45.787587 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:45.787589 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:45.787590 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:45.787739 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:45.787806 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:45.788177 (+ 371us) server_negotiation.cc:299] Negotiation successful
0811 18:57:45.788285 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 18:57:45.789147 22780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:45.778041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:40989 (local address 127.27.180.131:48465)
0811 18:57:45.778492 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:45.778532 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:45.779761 (+ 1229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:45.780347 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:45.780354 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:45.780781 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:57:45.781307 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:45.781323 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:45.782167 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:45.782171 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:45.783080 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:45.783089 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:45.783553 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:45.783556 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:45.783698 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:45.784169 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:57:45.784184 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:57:45.785581 (+ 1397us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:57:45.786943 (+ 1362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:45.786947 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:45.786958 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:45.787186 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:45.787404 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:45.787406 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:45.787407 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:45.787526 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:45.787805 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:57:45.787809 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:57:45.788006 (+ 197us) client_negotiation.cc:769] Sending connection context
0811 18:57:45.788146 (+ 140us) client_negotiation.cc:240] Negotiation successful
0811 18:57:45.788364 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":108,"threads_started":1}
I20260811 18:57:45.790143 22776 heartbeater.cc:347] Connected to a master server at 127.27.180.190:40989
I20260811 18:57:45.790371 22776 heartbeater.cc:464] Registering TS with master...
I20260811 18:57:45.790804 22776 heartbeater.cc:511] Master 127.27.180.190:40989 requested a full tablet report, sending...
I20260811 18:57:45.791750 21937 ts_manager.cc:194] Registered new tserver with Master: 1efdcbeb44e94a3bb3acb45daa0e7197 (127.27.180.131:45203)
I20260811 18:57:45.792290 21937 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.180.131@KRBTEST.COM'} at 127.27.180.131:48465
I20260811 18:57:45.793787 28370 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 18:57:45.802412 22005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:45.795561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58908 (local address 127.27.180.190:40989)
0811 18:57:45.795661 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:45.795665 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:45.795847 (+ 182us) server_negotiation.cc:407] Connection header received
0811 18:57:45.795958 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:45.795961 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:45.796001 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:45.796061 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:45.796704 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:45.797225 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:45.798016 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:45.798230 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:45.798322 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:45.799443 (+ 1121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:45.799446 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:45.799449 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:45.799472 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:45.800752 (+ 1280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:45.801193 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:45.801196 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:45.801198 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:45.801308 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:45.801564 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:45.801567 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:45.801569 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:45.801789 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:45.801902 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:45.802159 (+ 257us) server_negotiation.cc:299] Negotiation successful
0811 18:57:45.802272 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260811 18:57:45.804840 21937 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58908:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:45.807762 21937 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:45.821874 22795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:45.816084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34898 (local address 127.27.180.130:36347)
0811 18:57:45.816786 (+ 702us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:45.816801 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:45.816827 (+ 26us) server_negotiation.cc:407] Connection header received
0811 18:57:45.816912 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:45.816918 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:45.817156 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:45.817333 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:57:45.817874 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:45.819049 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:45.820495 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:45.821209 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:45.821339 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:45.821472 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:45.821646 (+ 174us) server_negotiation.cc:299] Negotiation successful
0811 18:57:45.821740 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":592,"thread_start_us":175,"threads_started":1}
I20260811 18:57:45.821851 22794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:45.815896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.130:36347 (local address 127.0.0.1:34898)
0811 18:57:45.816167 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:45.816209 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:45.816325 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:45.817524 (+ 1199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:45.817531 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:45.817557 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:57:45.817709 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:45.817718 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:45.819229 (+ 1511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:45.819233 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:45.820350 (+ 1117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:45.820361 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:45.821349 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:45.821353 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:45.821456 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:45.821491 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 18:57:45.821596 (+ 105us) client_negotiation.cc:240] Negotiation successful
0811 18:57:45.821710 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"mutex_wait_us":36,"thread_start_us":56,"threads_started":1}
I20260811 18:57:45.822207 22798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:45.816767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37614 (local address 127.27.180.131:45203)
0811 18:57:45.817685 (+ 918us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:45.817696 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:45.817715 (+ 19us) server_negotiation.cc:407] Connection header received
0811 18:57:45.817780 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:45.817785 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:45.817950 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:45.818081 (+ 131us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:57:45.818329 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:45.819027 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:45.820499 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:45.821264 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:45.821373 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:45.821493 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:45.821940 (+ 447us) server_negotiation.cc:299] Negotiation successful
0811 18:57:45.822041 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":824,"thread_start_us":119,"threads_started":1}
I20260811 18:57:45.822332 22797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:45.816582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.129:41443 (local address 127.0.0.1:38762)
0811 18:57:45.817074 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:45.817124 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:45.817212 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:45.817905 (+ 693us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:45.817908 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:45.817932 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:57:45.818033 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:45.818041 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:45.819578 (+ 1537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:45.819581 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:45.820866 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:45.820876 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:45.821955 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:45.821959 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:45.822056 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:45.822080 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 18:57:45.822135 (+ 55us) client_negotiation.cc:240] Negotiation successful
0811 18:57:45.822199 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":432,"mutex_wait_us":2,"thread_start_us":99,"threads_started":1}
I20260811 18:57:45.822577 22799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:45.816584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.131:45203 (local address 127.0.0.1:37614)
0811 18:57:45.817163 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:45.817174 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:45.817236 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:45.818101 (+ 865us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:45.818104 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:45.818122 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:57:45.818221 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:45.818228 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:45.819151 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:45.819154 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:45.820370 (+ 1216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:45.820379 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:45.821558 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:45.821563 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:45.821660 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:45.821689 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 18:57:45.821776 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 18:57:45.821854 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":526,"mutex_wait_us":203,"thread_start_us":102,"threads_started":1}
I20260811 18:57:45.823045 22801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:45.816780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38762 (local address 127.27.180.129:41443)
0811 18:57:45.817468 (+ 688us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:45.817480 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:45.817502 (+ 22us) server_negotiation.cc:407] Connection header received
0811 18:57:45.817576 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:45.817581 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:45.817757 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:45.817910 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:57:45.818193 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:45.819445 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:45.821000 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:45.821816 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:45.822601 (+ 785us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:45.822748 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:45.822805 (+ 57us) server_negotiation.cc:299] Negotiation successful
0811 18:57:45.822893 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":584,"thread_start_us":111,"threads_started":1}
I20260811 18:57:45.825304 22656 tablet_service.cc:1467] Processing CreateTablet for tablet d0efa96143d74ee8b986a5143df2e038 (DEFAULT_TABLE table=test-table [id=550be8bb25cf4e20939aae1a59d83bce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:45.825209 22414 tablet_service.cc:1467] Processing CreateTablet for tablet d0efa96143d74ee8b986a5143df2e038 (DEFAULT_TABLE table=test-table [id=550be8bb25cf4e20939aae1a59d83bce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:45.825604 22157 tablet_service.cc:1467] Processing CreateTablet for tablet d0efa96143d74ee8b986a5143df2e038 (DEFAULT_TABLE table=test-table [id=550be8bb25cf4e20939aae1a59d83bce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:45.826387 22656 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0efa96143d74ee8b986a5143df2e038. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:45.826380 22414 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0efa96143d74ee8b986a5143df2e038. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:45.826392 22157 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0efa96143d74ee8b986a5143df2e038. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:45.832069 22804 tablet_bootstrap.cc:492] T d0efa96143d74ee8b986a5143df2e038 P caf849fdfdc048128ec5b5502845312a: Bootstrap starting.
I20260811 18:57:45.834028 22805 tablet_bootstrap.cc:492] T d0efa96143d74ee8b986a5143df2e038 P 1efdcbeb44e94a3bb3acb45daa0e7197: Bootstrap starting.
I20260811 18:57:45.834412 22806 tablet_bootstrap.cc:492] T d0efa96143d74ee8b986a5143df2e038 P d37ed58d1e4e4273a7ad3a2a20681c90: Bootstrap starting.
I20260811 18:57:45.834712 22804 tablet_bootstrap.cc:654] T d0efa96143d74ee8b986a5143df2e038 P caf849fdfdc048128ec5b5502845312a: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:45.836172 22804 log.cc:824] T d0efa96143d74ee8b986a5143df2e038 P caf849fdfdc048128ec5b5502845312a: Log is configured to *not* fsync() on all Append() calls
I20260811 18:57:45.836632 22806 tablet_bootstrap.cc:654] T d0efa96143d74ee8b986a5143df2e038 P d37ed58d1e4e4273a7ad3a2a20681c90: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:45.836695 22805 tablet_bootstrap.cc:654] T d0efa96143d74ee8b986a5143df2e038 P 1efdcbeb44e94a3bb3acb45daa0e7197: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:45.837476 22806 log.cc:824] T d0efa96143d74ee8b986a5143df2e038 P d37ed58d1e4e4273a7ad3a2a20681c90: Log is configured to *not* fsync() on all Append() calls
I20260811 18:57:45.837488 22805 log.cc:824] T d0efa96143d74ee8b986a5143df2e038 P 1efdcbeb44e94a3bb3acb45daa0e7197: Log is configured to *not* fsync() on all Append() calls
I20260811 18:57:45.838568 22804 tablet_bootstrap.cc:492] T d0efa96143d74ee8b986a5143df2e038 P caf849fdfdc048128ec5b5502845312a: No bootstrap required, opened a new log
I20260811 18:57:45.838743 22804 ts_tablet_manager.cc:1397] T d0efa96143d74ee8b986a5143df2e038 P caf849fdfdc048128ec5b5502845312a: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260811 18:57:45.839386 22806 tablet_bootstrap.cc:492] T d0efa96143d74ee8b986a5143df2e038 P d37ed58d1e4e4273a7ad3a2a20681c90: No bootstrap required, opened a new log
I20260811 18:57:45.839563 22806 ts_tablet_manager.cc:1397] T d0efa96143d74ee8b986a5143df2e038 P d37ed58d1e4e4273a7ad3a2a20681c90: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 18:57:45.841661 22804 raft_consensus.cc:348] T d0efa96143d74ee8b986a5143df2e038 P caf849fdfdc048128ec5b5502845312a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d37ed58d1e4e4273a7ad3a2a20681c90" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 36347 } } peers { permanent_uuid: "1efdcbeb44e94a3bb3acb45daa0e7197" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 45203 } } peers { permanent_uuid: "caf849fdfdc048128ec5b5502845312a" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 41443 } }
I20260811 18:57:45.841872 22804 raft_consensus.cc:374] T d0efa96143d74ee8b986a5143df2e038 P caf849fdfdc048128ec5b5502845312a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:45.841915 22804 raft_consensus.cc:729] T d0efa96143d74ee8b986a5143df2e038 P caf849fdfdc048128ec5b5502845312a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: caf849fdfdc048128ec5b5502845312a, State: Initialized, Role: FOLLOWER
I20260811 18:57:45.842300 22804 consensus_queue.cc:260] T d0efa96143d74ee8b986a5143df2e038 P caf849fdfdc048128ec5b5502845312a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d37ed58d1e4e4273a7ad3a2a20681c90" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 36347 } } peers { permanent_uuid: "1efdcbeb44e94a3bb3acb45daa0e7197" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 45203 } } peers { permanent_uuid: "caf849fdfdc048128ec5b5502845312a" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 41443 } }
I20260811 18:57:45.842562 22806 raft_consensus.cc:348] T d0efa96143d74ee8b986a5143df2e038 P d37ed58d1e4e4273a7ad3a2a20681c90 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d37ed58d1e4e4273a7ad3a2a20681c90" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 36347 } } peers { permanent_uuid: "1efdcbeb44e94a3bb3acb45daa0e7197" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 45203 } } peers { permanent_uuid: "caf849fdfdc048128ec5b5502845312a" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 41443 } }
I20260811 18:57:45.842769 22806 raft_consensus.cc:374] T d0efa96143d74ee8b986a5143df2e038 P d37ed58d1e4e4273a7ad3a2a20681c90 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:45.842856 22806 raft_consensus.cc:729] T d0efa96143d74ee8b986a5143df2e038 P d37ed58d1e4e4273a7ad3a2a20681c90 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d37ed58d1e4e4273a7ad3a2a20681c90, State: Initialized, Role: FOLLOWER
I20260811 18:57:45.842893 22286 heartbeater.cc:503] Master 127.27.180.190:40989 was elected leader, sending a full tablet report...
I20260811 18:57:45.843245 22804 ts_tablet_manager.cc:1428] T d0efa96143d74ee8b986a5143df2e038 P caf849fdfdc048128ec5b5502845312a: Time spent starting tablet: real 0.004s user 0.001s sys 0.003s
I20260811 18:57:45.843281 22806 consensus_queue.cc:260] T d0efa96143d74ee8b986a5143df2e038 P d37ed58d1e4e4273a7ad3a2a20681c90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d37ed58d1e4e4273a7ad3a2a20681c90" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 36347 } } peers { permanent_uuid: "1efdcbeb44e94a3bb3acb45daa0e7197" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 45203 } } peers { permanent_uuid: "caf849fdfdc048128ec5b5502845312a" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 41443 } }
I20260811 18:57:45.844321 22806 ts_tablet_manager.cc:1428] T d0efa96143d74ee8b986a5143df2e038 P d37ed58d1e4e4273a7ad3a2a20681c90: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260811 18:57:45.844749 22540 heartbeater.cc:503] Master 127.27.180.190:40989 was elected leader, sending a full tablet report...
I20260811 18:57:45.845604 22805 tablet_bootstrap.cc:492] T d0efa96143d74ee8b986a5143df2e038 P 1efdcbeb44e94a3bb3acb45daa0e7197: No bootstrap required, opened a new log
I20260811 18:57:45.845788 22805 ts_tablet_manager.cc:1397] T d0efa96143d74ee8b986a5143df2e038 P 1efdcbeb44e94a3bb3acb45daa0e7197: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.000s
I20260811 18:57:45.848753 22805 raft_consensus.cc:348] T d0efa96143d74ee8b986a5143df2e038 P 1efdcbeb44e94a3bb3acb45daa0e7197 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d37ed58d1e4e4273a7ad3a2a20681c90" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 36347 } } peers { permanent_uuid: "1efdcbeb44e94a3bb3acb45daa0e7197" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 45203 } } peers { permanent_uuid: "caf849fdfdc048128ec5b5502845312a" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 41443 } }
I20260811 18:57:45.848946 22805 raft_consensus.cc:374] T d0efa96143d74ee8b986a5143df2e038 P 1efdcbeb44e94a3bb3acb45daa0e7197 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:45.849013 22805 raft_consensus.cc:729] T d0efa96143d74ee8b986a5143df2e038 P 1efdcbeb44e94a3bb3acb45daa0e7197 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1efdcbeb44e94a3bb3acb45daa0e7197, State: Initialized, Role: FOLLOWER
I20260811 18:57:45.849400 22805 consensus_queue.cc:260] T d0efa96143d74ee8b986a5143df2e038 P 1efdcbeb44e94a3bb3acb45daa0e7197 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d37ed58d1e4e4273a7ad3a2a20681c90" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 36347 } } peers { permanent_uuid: "1efdcbeb44e94a3bb3acb45daa0e7197" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 45203 } } peers { permanent_uuid: "caf849fdfdc048128ec5b5502845312a" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 41443 } }
I20260811 18:57:45.849988 22776 heartbeater.cc:503] Master 127.27.180.190:40989 was elected leader, sending a full tablet report...
I20260811 18:57:45.850215 22805 ts_tablet_manager.cc:1428] T d0efa96143d74ee8b986a5143df2e038 P 1efdcbeb44e94a3bb3acb45daa0e7197: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
W20260811 18:57:45.858053 22287 tablet.cc:2369] T d0efa96143d74ee8b986a5143df2e038 P caf849fdfdc048128ec5b5502845312a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:57:45.968995 22816 raft_consensus.cc:482] T d0efa96143d74ee8b986a5143df2e038 P d37ed58d1e4e4273a7ad3a2a20681c90 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:45.969326 22816 raft_consensus.cc:504] T d0efa96143d74ee8b986a5143df2e038 P d37ed58d1e4e4273a7ad3a2a20681c90 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d37ed58d1e4e4273a7ad3a2a20681c90" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 36347 } } peers { permanent_uuid: "1efdcbeb44e94a3bb3acb45daa0e7197" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 45203 } } peers { permanent_uuid: "caf849fdfdc048128ec5b5502845312a" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 41443 } }
I20260811 18:57:45.970770 22816 leader_election.cc:288] T d0efa96143d74ee8b986a5143df2e038 P d37ed58d1e4e4273a7ad3a2a20681c90 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1efdcbeb44e94a3bb3acb45daa0e7197 (127.27.180.131:45203), caf849fdfdc048128ec5b5502845312a (127.27.180.129:41443)
I20260811 18:57:45.974764 22801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:45.971160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.130:56431 (local address 127.27.180.129:41443)
0811 18:57:45.971309 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:45.971314 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:45.971612 (+ 298us) server_negotiation.cc:407] Connection header received
0811 18:57:45.971714 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:45.971717 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:45.971767 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:45.971861 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:57:45.972122 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:45.972617 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:45.973797 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:45.974214 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:45.974329 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:45.974388 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:45.974557 (+ 169us) server_negotiation.cc:299] Negotiation successful
0811 18:57:45.974633 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 18:57:45.974771 22798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:45.971310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.130:58607 (local address 127.27.180.131:45203)
0811 18:57:45.971446 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:45.971450 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:45.971465 (+ 15us) server_negotiation.cc:407] Connection header received
0811 18:57:45.971522 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:45.971525 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:45.971575 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:45.971651 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:57:45.971954 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:45.972563 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:45.973807 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:45.974218 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:45.974335 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:45.974387 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:45.974582 (+ 195us) server_negotiation.cc:299] Negotiation successful
0811 18:57:45.974649 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 18:57:45.974764 22542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:45.971001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.131:45203 (local address 127.27.180.130:58607)
0811 18:57:45.971119 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:45.971135 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:45.971219 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:45.971678 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:45.971681 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:45.971706 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:57:45.971835 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:45.971842 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:45.972677 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:45.972681 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:45.973672 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:45.973683 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:45.974342 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:45.974345 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:45.974461 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:45.974478 (+ 17us) client_negotiation.cc:769] Sending connection context
0811 18:57:45.974551 (+ 73us) client_negotiation.cc:240] Negotiation successful
0811 18:57:45.974621 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":87}
I20260811 18:57:45.974788 22819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:45.971004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.129:41443 (local address 127.27.180.130:56431)
0811 18:57:45.971505 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:45.971517 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:45.971621 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:45.971866 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:45.971868 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:45.971882 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:57:45.972017 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:45.972025 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:45.972729 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:45.972739 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:45.973666 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:45.973677 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:45.974335 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:45.974339 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:45.974433 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:45.974457 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 18:57:45.974516 (+ 59us) client_negotiation.cc:240] Negotiation successful
0811 18:57:45.974585 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":441,"mutex_wait_us":138,"thread_start_us":119,"threads_started":1}
I20260811 18:57:45.975384 22192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0efa96143d74ee8b986a5143df2e038" candidate_uuid: "d37ed58d1e4e4273a7ad3a2a20681c90" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "caf849fdfdc048128ec5b5502845312a" is_pre_election: true
I20260811 18:57:45.975551 22697 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0efa96143d74ee8b986a5143df2e038" candidate_uuid: "d37ed58d1e4e4273a7ad3a2a20681c90" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efdcbeb44e94a3bb3acb45daa0e7197" is_pre_election: true
I20260811 18:57:45.975688 22192 raft_consensus.cc:2454] T d0efa96143d74ee8b986a5143df2e038 P caf849fdfdc048128ec5b5502845312a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d37ed58d1e4e4273a7ad3a2a20681c90 in term 0.
I20260811 18:57:45.975852 22697 raft_consensus.cc:2454] T d0efa96143d74ee8b986a5143df2e038 P 1efdcbeb44e94a3bb3acb45daa0e7197 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d37ed58d1e4e4273a7ad3a2a20681c90 in term 0.
I20260811 18:57:45.976188 22329 leader_election.cc:302] T d0efa96143d74ee8b986a5143df2e038 P d37ed58d1e4e4273a7ad3a2a20681c90 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: caf849fdfdc048128ec5b5502845312a, d37ed58d1e4e4273a7ad3a2a20681c90; no voters:
I20260811 18:57:45.976457 22816 raft_consensus.cc:2781] T d0efa96143d74ee8b986a5143df2e038 P d37ed58d1e4e4273a7ad3a2a20681c90 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:45.976547 22816 raft_consensus.cc:482] T d0efa96143d74ee8b986a5143df2e038 P d37ed58d1e4e4273a7ad3a2a20681c90 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:45.976590 22816 raft_consensus.cc:3037] T d0efa96143d74ee8b986a5143df2e038 P d37ed58d1e4e4273a7ad3a2a20681c90 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:45.977707 22816 raft_consensus.cc:504] T d0efa96143d74ee8b986a5143df2e038 P d37ed58d1e4e4273a7ad3a2a20681c90 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d37ed58d1e4e4273a7ad3a2a20681c90" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 36347 } } peers { permanent_uuid: "1efdcbeb44e94a3bb3acb45daa0e7197" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 45203 } } peers { permanent_uuid: "caf849fdfdc048128ec5b5502845312a" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 41443 } }
I20260811 18:57:45.978030 22816 leader_election.cc:288] T d0efa96143d74ee8b986a5143df2e038 P d37ed58d1e4e4273a7ad3a2a20681c90 [CANDIDATE]: Term 1 election: Requested vote from peers 1efdcbeb44e94a3bb3acb45daa0e7197 (127.27.180.131:45203), caf849fdfdc048128ec5b5502845312a (127.27.180.129:41443)
I20260811 18:57:45.978470 22697 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0efa96143d74ee8b986a5143df2e038" candidate_uuid: "d37ed58d1e4e4273a7ad3a2a20681c90" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efdcbeb44e94a3bb3acb45daa0e7197"
I20260811 18:57:45.978619 22697 raft_consensus.cc:3037] T d0efa96143d74ee8b986a5143df2e038 P 1efdcbeb44e94a3bb3acb45daa0e7197 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:45.979689 22697 raft_consensus.cc:2454] T d0efa96143d74ee8b986a5143df2e038 P 1efdcbeb44e94a3bb3acb45daa0e7197 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d37ed58d1e4e4273a7ad3a2a20681c90 in term 1.
I20260811 18:57:45.979993 22192 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0efa96143d74ee8b986a5143df2e038" candidate_uuid: "d37ed58d1e4e4273a7ad3a2a20681c90" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "caf849fdfdc048128ec5b5502845312a"
I20260811 18:57:45.980047 22339 leader_election.cc:302] T d0efa96143d74ee8b986a5143df2e038 P d37ed58d1e4e4273a7ad3a2a20681c90 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1efdcbeb44e94a3bb3acb45daa0e7197, d37ed58d1e4e4273a7ad3a2a20681c90; no voters:
I20260811 18:57:45.980144 22192 raft_consensus.cc:3037] T d0efa96143d74ee8b986a5143df2e038 P caf849fdfdc048128ec5b5502845312a [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:45.980226 22816 raft_consensus.cc:2781] T d0efa96143d74ee8b986a5143df2e038 P d37ed58d1e4e4273a7ad3a2a20681c90 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:45.980471 22816 raft_consensus.cc:686] T d0efa96143d74ee8b986a5143df2e038 P d37ed58d1e4e4273a7ad3a2a20681c90 [term 1 LEADER]: Becoming Leader. State: Replica: d37ed58d1e4e4273a7ad3a2a20681c90, State: Running, Role: LEADER
I20260811 18:57:45.980813 22816 consensus_queue.cc:237] T d0efa96143d74ee8b986a5143df2e038 P d37ed58d1e4e4273a7ad3a2a20681c90 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d37ed58d1e4e4273a7ad3a2a20681c90" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 36347 } } peers { permanent_uuid: "1efdcbeb44e94a3bb3acb45daa0e7197" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 45203 } } peers { permanent_uuid: "caf849fdfdc048128ec5b5502845312a" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 41443 } }
I20260811 18:57:45.981415 22192 raft_consensus.cc:2454] T d0efa96143d74ee8b986a5143df2e038 P caf849fdfdc048128ec5b5502845312a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d37ed58d1e4e4273a7ad3a2a20681c90 in term 1.
I20260811 18:57:45.984208 21937 catalog_manager.cc:5495] T d0efa96143d74ee8b986a5143df2e038 P d37ed58d1e4e4273a7ad3a2a20681c90 reported cstate change: term changed from 0 to 1, leader changed from <none> to d37ed58d1e4e4273a7ad3a2a20681c90 (127.27.180.130). New cstate: current_term: 1 leader_uuid: "d37ed58d1e4e4273a7ad3a2a20681c90" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d37ed58d1e4e4273a7ad3a2a20681c90" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 36347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efdcbeb44e94a3bb3acb45daa0e7197" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 45203 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "caf849fdfdc048128ec5b5502845312a" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 41443 } health_report { overall_health: UNKNOWN } } }
I20260811 18:57:46.016112 22795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:46.012912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34904 (local address 127.27.180.130:36347)
0811 18:57:46.013033 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:46.013037 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:46.013050 (+ 13us) server_negotiation.cc:407] Connection header received
0811 18:57:46.013142 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:46.013144 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:46.013202 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:46.013299 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:57:46.013515 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:46.014020 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:46.014843 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:46.015066 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:46.015149 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:46.015384 (+ 235us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:57:46.015806 (+ 422us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:57:46.015897 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:46.015927 (+ 30us) server_negotiation.cc:299] Negotiation successful
0811 18:57:46.015980 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 18:57:46.022023 22192 raft_consensus.cc:1261] T d0efa96143d74ee8b986a5143df2e038 P caf849fdfdc048128ec5b5502845312a [term 1 FOLLOWER]: Refusing update from remote peer d37ed58d1e4e4273a7ad3a2a20681c90: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:57:46.022027 22697 raft_consensus.cc:1261] T d0efa96143d74ee8b986a5143df2e038 P 1efdcbeb44e94a3bb3acb45daa0e7197 [term 1 FOLLOWER]: Refusing update from remote peer d37ed58d1e4e4273a7ad3a2a20681c90: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:57:46.022727 22823 consensus_queue.cc:1035] T d0efa96143d74ee8b986a5143df2e038 P d37ed58d1e4e4273a7ad3a2a20681c90 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efdcbeb44e94a3bb3acb45daa0e7197" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 45203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:46.022898 22816 consensus_queue.cc:1035] T d0efa96143d74ee8b986a5143df2e038 P d37ed58d1e4e4273a7ad3a2a20681c90 [LEADER]: Connected to new peer: Peer: permanent_uuid: "caf849fdfdc048128ec5b5502845312a" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 41443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 18:57:46.026870 22777 tablet.cc:2369] T d0efa96143d74ee8b986a5143df2e038 P 1efdcbeb44e94a3bb3acb45daa0e7197: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:57:46.032601 22836 mvcc.cc:205] Tried to move back new op lower bound from 7317400232018735104 to 7317400231860236288. Current Snapshot: MvccSnapshot[applied={T|T < 7317400232018735104 or (T in {7317400232018735104})}]
I20260811 18:57:46.038443 21934 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58908:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:57:46.038587 21934 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58908:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:57:46.040860 21934 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 46e6feef897141feb6bf6aba7a68a7ce: Sending DeleteTablet for 3 replicas of tablet d0efa96143d74ee8b986a5143df2e038
I20260811 18:57:46.041687 22414 tablet_service.cc:1514] Processing DeleteTablet for tablet d0efa96143d74ee8b986a5143df2e038 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:46 UTC) from {username='kudu', principal='kudu/127.27.180.190@KRBTEST.COM'} at 127.0.0.1:34898
I20260811 18:57:46.041751 22157 tablet_service.cc:1514] Processing DeleteTablet for tablet d0efa96143d74ee8b986a5143df2e038 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:46 UTC) from {username='kudu', principal='kudu/127.27.180.190@KRBTEST.COM'} at 127.0.0.1:38762
I20260811 18:57:46.041942 22656 tablet_service.cc:1514] Processing DeleteTablet for tablet d0efa96143d74ee8b986a5143df2e038 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:46 UTC) from {username='kudu', principal='kudu/127.27.180.190@KRBTEST.COM'} at 127.0.0.1:37614
I20260811 18:57:46.042156 22864 tablet_replica.cc:330] T d0efa96143d74ee8b986a5143df2e038 P d37ed58d1e4e4273a7ad3a2a20681c90: stopping tablet replica
I20260811 18:57:46.042325 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 22049
I20260811 18:57:46.056492 22864 raft_consensus.cc:2229] T d0efa96143d74ee8b986a5143df2e038 P d37ed58d1e4e4273a7ad3a2a20681c90 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:46.057055 22864 raft_consensus.cc:2258] T d0efa96143d74ee8b986a5143df2e038 P d37ed58d1e4e4273a7ad3a2a20681c90 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:46.058918 22864 ts_tablet_manager.cc:1905] T d0efa96143d74ee8b986a5143df2e038 P d37ed58d1e4e4273a7ad3a2a20681c90: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:46.061296 22868 tablet_replica.cc:330] T d0efa96143d74ee8b986a5143df2e038 P 1efdcbeb44e94a3bb3acb45daa0e7197: stopping tablet replica
I20260811 18:57:46.061609 22868 raft_consensus.cc:2229] T d0efa96143d74ee8b986a5143df2e038 P 1efdcbeb44e94a3bb3acb45daa0e7197 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:46.061827 22864 ts_tablet_manager.cc:1918] T d0efa96143d74ee8b986a5143df2e038 P d37ed58d1e4e4273a7ad3a2a20681c90: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:57:46.061940 22864 log.cc:1196] T d0efa96143d74ee8b986a5143df2e038 P d37ed58d1e4e4273a7ad3a2a20681c90: Deleting WAL directory at /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-1/wal/wals/d0efa96143d74ee8b986a5143df2e038
I20260811 18:57:46.061923 22868 raft_consensus.cc:2258] T d0efa96143d74ee8b986a5143df2e038 P 1efdcbeb44e94a3bb3acb45daa0e7197 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:46.062296 22864 ts_tablet_manager.cc:1939] T d0efa96143d74ee8b986a5143df2e038 P d37ed58d1e4e4273a7ad3a2a20681c90: Deleting consensus metadata
I20260811 18:57:46.063541 21908 catalog_manager.cc:4841] TS d37ed58d1e4e4273a7ad3a2a20681c90 (127.27.180.130:36347): tablet d0efa96143d74ee8b986a5143df2e038 (table test-table [id=550be8bb25cf4e20939aae1a59d83bce]) successfully deleted
I20260811 18:57:46.064919 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 22295
W20260811 18:57:46.064962 21906 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.27.180.129:41443 (error 108)
I20260811 18:57:46.076404 22799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:46.076089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.129:41443 (local address 127.0.0.1:38774)
0811 18:57:46.076220 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:46.076306 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.129:41443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260811 18:57:46.076584 21906 catalog_manager.cc:4584] TS caf849fdfdc048128ec5b5502845312a (127.27.180.129:41443): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d0efa96143d74ee8b986a5143df2e038: Network error: Client connection negotiation failed: client connection to 127.27.180.129:41443: connect: Connection refused (error 111)
I20260811 18:57:46.077545 22868 ts_tablet_manager.cc:1905] T d0efa96143d74ee8b986a5143df2e038 P 1efdcbeb44e94a3bb3acb45daa0e7197: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:46.080185 22868 ts_tablet_manager.cc:1918] T d0efa96143d74ee8b986a5143df2e038 P 1efdcbeb44e94a3bb3acb45daa0e7197: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:57:46.080296 22868 log.cc:1196] T d0efa96143d74ee8b986a5143df2e038 P 1efdcbeb44e94a3bb3acb45daa0e7197: Deleting WAL directory at /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-2/wal/wals/d0efa96143d74ee8b986a5143df2e038
I20260811 18:57:46.080614 22868 ts_tablet_manager.cc:1939] T d0efa96143d74ee8b986a5143df2e038 P 1efdcbeb44e94a3bb3acb45daa0e7197: Deleting consensus metadata
I20260811 18:57:46.081914 21913 catalog_manager.cc:4841] TS 1efdcbeb44e94a3bb3acb45daa0e7197 (127.27.180.131:45203): tablet d0efa96143d74ee8b986a5143df2e038 (table test-table [id=550be8bb25cf4e20939aae1a59d83bce]) successfully deleted
I20260811 18:57:46.085728 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 22551
I20260811 18:57:46.096386 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 21879
I20260811 18:57:46.106776 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.190:40989
--webserver_interface=127.27.180.190
--webserver_port=37621
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab
--principal=kudu/127.27.180.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:42327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.180.190:40989
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:57:46.216642 22870 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:57:46.216898 22870 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:57:46.216938 22870 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:57:46.220875 22870 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:57:46.220929 22870 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:57:46.220947 22870 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:57:46.220961 22870 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:57:46.220974 22870 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:57:46.226135 22870 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:42327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.27.180.190:40989
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab
--principal=kudu/127.27.180.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.190:40989
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.180.190
--webserver_port=37621
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22870
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:57:46.227140 22870 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:57:46.228149 22870 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:57:46.241760 22885 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:46.241874 22883 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:57:46.241922 22870 server_base.cc:1034] running on GCE node
W20260811 18:57:46.242148 22889 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:57:46.242777 22870 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:57:46.243883 22870 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:57:46.245064 22870 hybrid_clock.cc:648] HybridClock initialized: now 1786474666245035 us; error 42 us; skew 500 ppm
Aug 11 18:57:46 dist-test-slave-2j7r krb5kdc[21860](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474666, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:57:46.247925 22870 init.cc:377] Logged in from keytab as kudu/127.27.180.190@KRBTEST.COM (short username kudu)
I20260811 18:57:46.248835 22870 webserver.cc:460] Webserver started at http://127.27.180.190:37621/ using document root <none> and password file <none>
I20260811 18:57:46.249477 22870 fs_manager.cc:359] Metadata directory not provided
I20260811 18:57:46.249554 22870 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:57:46.253451 22870 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260811 18:57:46.255650 22908 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:46.256914 22870 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 18:57:46.257035 22870 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/master-0/wal
uuid: "46e6feef897141feb6bf6aba7a68a7ce"
format_stamp: "Formatted at 2026-08-11 18:57:45 on dist-test-slave-2j7r"
server_key: "446f7ff2aba1006e44a2e51af698b0e1"
server_key_iv: "409541b09efda84ff9e45bf5477e5af7"
server_key_version: "encryptionkey@0"
I20260811 18:57:46.257673 22870 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:57:46.275167 22870 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:57:46.278096 22870 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:57:46.278308 22870 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:57:46.285331 22870 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.190:40989
I20260811 18:57:46.285594 23009 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.190:40989 every 8 connection(s)
I20260811 18:57:46.286438 22870 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
I20260811 18:57:46.290756 23011 sys_catalog.cc:263] Verifying existing consensus state
I20260811 18:57:46.292402 23011 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46e6feef897141feb6bf6aba7a68a7ce: Bootstrap starting.
I20260811 18:57:46.293283 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 22870
I20260811 18:57:46.293695 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.180.129:41443
--local_ip_for_outbound_sockets=127.27.180.129
--tserver_master_addrs=127.27.180.190:40989
--webserver_port=46263
--webserver_interface=127.27.180.129
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.129.keytab
--principal=kudu/127.27.180.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:42327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 18:57:46.311172 23011 log.cc:824] T 00000000000000000000000000000000 P 46e6feef897141feb6bf6aba7a68a7ce: Log is configured to *not* fsync() on all Append() calls
I20260811 18:57:46.320461 23011 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46e6feef897141feb6bf6aba7a68a7ce: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 18:57:46.320822 23011 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46e6feef897141feb6bf6aba7a68a7ce: Bootstrap complete.
I20260811 18:57:46.324872 23011 raft_consensus.cc:348] T 00000000000000000000000000000000 P 46e6feef897141feb6bf6aba7a68a7ce [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46e6feef897141feb6bf6aba7a68a7ce" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 40989 } }
I20260811 18:57:46.325320 23011 raft_consensus.cc:729] T 00000000000000000000000000000000 P 46e6feef897141feb6bf6aba7a68a7ce [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46e6feef897141feb6bf6aba7a68a7ce, State: Initialized, Role: FOLLOWER
I20260811 18:57:46.325804 23011 consensus_queue.cc:260] T 00000000000000000000000000000000 P 46e6feef897141feb6bf6aba7a68a7ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46e6feef897141feb6bf6aba7a68a7ce" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 40989 } }
I20260811 18:57:46.325951 23011 raft_consensus.cc:388] T 00000000000000000000000000000000 P 46e6feef897141feb6bf6aba7a68a7ce [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:57:46.326026 23011 raft_consensus.cc:482] T 00000000000000000000000000000000 P 46e6feef897141feb6bf6aba7a68a7ce [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:57:46.326124 23011 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 46e6feef897141feb6bf6aba7a68a7ce [term 1 FOLLOWER]: Advancing to term 2
I20260811 18:57:46.327975 23011 raft_consensus.cc:504] T 00000000000000000000000000000000 P 46e6feef897141feb6bf6aba7a68a7ce [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46e6feef897141feb6bf6aba7a68a7ce" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 40989 } }
I20260811 18:57:46.328293 23011 leader_election.cc:302] T 00000000000000000000000000000000 P 46e6feef897141feb6bf6aba7a68a7ce [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 46e6feef897141feb6bf6aba7a68a7ce; no voters:
I20260811 18:57:46.328588 23011 leader_election.cc:288] T 00000000000000000000000000000000 P 46e6feef897141feb6bf6aba7a68a7ce [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 18:57:46.328694 23020 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 46e6feef897141feb6bf6aba7a68a7ce [term 2 FOLLOWER]: Leader election won for term 2
I20260811 18:57:46.328945 23020 raft_consensus.cc:686] T 00000000000000000000000000000000 P 46e6feef897141feb6bf6aba7a68a7ce [term 2 LEADER]: Becoming Leader. State: Replica: 46e6feef897141feb6bf6aba7a68a7ce, State: Running, Role: LEADER
I20260811 18:57:46.329324 23020 consensus_queue.cc:237] T 00000000000000000000000000000000 P 46e6feef897141feb6bf6aba7a68a7ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46e6feef897141feb6bf6aba7a68a7ce" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 40989 } }
I20260811 18:57:46.329907 23011 sys_catalog.cc:564] T 00000000000000000000000000000000 P 46e6feef897141feb6bf6aba7a68a7ce [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:57:46.330674 23022 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46e6feef897141feb6bf6aba7a68a7ce [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "46e6feef897141feb6bf6aba7a68a7ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46e6feef897141feb6bf6aba7a68a7ce" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 40989 } } }
I20260811 18:57:46.330814 23022 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46e6feef897141feb6bf6aba7a68a7ce [sys.catalog]: This master's current role is: LEADER
I20260811 18:57:46.331194 23024 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46e6feef897141feb6bf6aba7a68a7ce [sys.catalog]: SysCatalogTable state changed. Reason: New leader 46e6feef897141feb6bf6aba7a68a7ce. Latest consensus state: current_term: 2 leader_uuid: "46e6feef897141feb6bf6aba7a68a7ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46e6feef897141feb6bf6aba7a68a7ce" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 40989 } } }
I20260811 18:57:46.331348 23024 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46e6feef897141feb6bf6aba7a68a7ce [sys.catalog]: This master's current role is: LEADER
I20260811 18:57:46.331673 23034 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:57:46.336472 23034 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:57:46.337184 23034 catalog_manager.cc:1259] Loaded cluster ID: fe5b4857e0284e22ba246b0656cb5426
I20260811 18:57:46.337244 23034 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:57:46.338660 23034 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260811 18:57:46.343174 23034 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260811 18:57:46.348544 23034 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:57:46.349362 23034 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 46e6feef897141feb6bf6aba7a68a7ce: Loaded TSK: 0
I20260811 18:57:46.350548 23034 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 18:57:46.412230 23015 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:57:46.412487 23015 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:57:46.412531 23015 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:57:46.416713 23015 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:57:46.416792 23015 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:57:46.416852 23015 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.180.129
I20260811 18:57:46.422175 23015 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:42327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.129.keytab
--principal=kudu/127.27.180.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.180.129:41443
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.180.129
--webserver_port=46263
--tserver_master_addrs=127.27.180.190:40989
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23015
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.180.129
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:57:46.423209 23015 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:57:46.424227 23015 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:57:46.434016 23056 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:46.434016 23060 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:46.434360 23064 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:57:46.434418 23015 server_base.cc:1034] running on GCE node
I20260811 18:57:46.434860 23015 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:57:46.435497 23015 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:57:46.436684 23015 hybrid_clock.cc:648] HybridClock initialized: now 1786474666436651 us; error 52 us; skew 500 ppm
Aug 11 18:57:46 dist-test-slave-2j7r krb5kdc[21860](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474666, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:57:46.439653 23015 init.cc:377] Logged in from keytab as kudu/127.27.180.129@KRBTEST.COM (short username kudu)
I20260811 18:57:46.440594 23015 webserver.cc:460] Webserver started at http://127.27.180.129:46263/ using document root <none> and password file <none>
I20260811 18:57:46.441190 23015 fs_manager.cc:359] Metadata directory not provided
I20260811 18:57:46.441277 23015 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:57:46.445379 23015 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:57:46.447644 23079 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:46.449175 23015 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:57:46.449334 23015 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-0/wal
uuid: "caf849fdfdc048128ec5b5502845312a"
format_stamp: "Formatted at 2026-08-11 18:57:45 on dist-test-slave-2j7r"
server_key: "9bf6192df9a7f04098127322b8a2ef92"
server_key_iv: "bc7f2259ae066343b8817f074423818a"
server_key_version: "encryptionkey@0"
I20260811 18:57:46.449888 23015 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:57:46.467700 23015 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:57:46.471014 23015 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:57:46.471215 23015 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:57:46.471848 23015 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:57:46.472959 23099 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 18:57:46.477394 23015 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 18:57:46.477480 23015 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20260811 18:57:46.477553 23015 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 18:57:46.479321 23015 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 18:57:46.479370 23015 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260811 18:57:46.479673 23099 tablet_bootstrap.cc:492] T d0efa96143d74ee8b986a5143df2e038 P caf849fdfdc048128ec5b5502845312a: Bootstrap starting.
I20260811 18:57:46.489049 23015 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.129:41443
I20260811 18:57:46.489187 23276 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.129:41443 every 8 connection(s)
I20260811 18:57:46.490077 23015 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-0/data/info.pb
I20260811 18:57:46.491362 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 23015
I20260811 18:57:46.491909 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.180.130:36347
--local_ip_for_outbound_sockets=127.27.180.130
--tserver_master_addrs=127.27.180.190:40989
--webserver_port=40669
--webserver_interface=127.27.180.130
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.130.keytab
--principal=kudu/127.27.180.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:42327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 18:57:46 dist-test-slave-2j7r krb5kdc[21860](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474666, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.129@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM
I20260811 18:57:46.500833 23099 log.cc:824] T d0efa96143d74ee8b986a5143df2e038 P caf849fdfdc048128ec5b5502845312a: Log is configured to *not* fsync() on all Append() calls
I20260811 18:57:46.507246 23280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:46.492083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:40989 (local address 127.27.180.129:51693)
0811 18:57:46.492910 (+ 827us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:46.492966 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:46.493906 (+ 940us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:46.495096 (+ 1190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:46.495104 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:46.495558 (+ 454us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:57:46.496020 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:46.496035 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:46.497356 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:46.497359 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:46.498260 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:46.498270 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:46.498977 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:46.498981 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:46.499164 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:46.499740 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:57:46.499757 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:57:46.501235 (+ 1478us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:57:46.503762 (+ 2527us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:46.503769 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:46.503781 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:46.504124 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:46.504533 (+ 409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:46.504536 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:46.504538 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:46.504697 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:46.505480 (+ 783us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:57:46.505486 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:57:46.505760 (+ 274us) client_negotiation.cc:769] Sending connection context
0811 18:57:46.505970 (+ 210us) client_negotiation.cc:240] Negotiation successful
0811 18:57:46.506299 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":624,"thread_start_us":129,"threads_started":1}
I20260811 18:57:46.507328 23279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:46.492068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.129:51693 (local address 127.27.180.190:40989)
0811 18:57:46.492606 (+ 538us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:46.492626 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:46.493125 (+ 499us) server_negotiation.cc:407] Connection header received
0811 18:57:46.494384 (+ 1259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:46.494396 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:46.494699 (+ 303us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:46.495048 (+ 349us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:46.496195 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:46.497206 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:46.498381 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:46.498839 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:46.498988 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:46.501512 (+ 2524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:46.501520 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:46.501538 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:46.501575 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:46.503619 (+ 2044us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:46.504253 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:46.504260 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:46.504267 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:46.504425 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:46.504818 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:46.504822 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:46.504824 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:46.505347 (+ 523us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:46.505485 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:46.506115 (+ 630us) server_negotiation.cc:299] Negotiation successful
0811 18:57:46.506447 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":396,"thread_start_us":142,"threads_started":1}
I20260811 18:57:46.508973 23099 tablet_bootstrap.cc:492] T d0efa96143d74ee8b986a5143df2e038 P caf849fdfdc048128ec5b5502845312a: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 18:57:46.509383 23099 tablet_bootstrap.cc:492] T d0efa96143d74ee8b986a5143df2e038 P caf849fdfdc048128ec5b5502845312a: Bootstrap complete.
I20260811 18:57:46.510339 23277 heartbeater.cc:347] Connected to a master server at 127.27.180.190:40989
I20260811 18:57:46.510562 23277 heartbeater.cc:464] Registering TS with master...
I20260811 18:57:46.510651 23099 ts_tablet_manager.cc:1397] T d0efa96143d74ee8b986a5143df2e038 P caf849fdfdc048128ec5b5502845312a: Time spent bootstrapping tablet: real 0.031s user 0.021s sys 0.008s
I20260811 18:57:46.511094 23277 heartbeater.cc:511] Master 127.27.180.190:40989 requested a full tablet report, sending...
I20260811 18:57:46.513338 23099 raft_consensus.cc:348] T d0efa96143d74ee8b986a5143df2e038 P caf849fdfdc048128ec5b5502845312a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d37ed58d1e4e4273a7ad3a2a20681c90" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 36347 } } peers { permanent_uuid: "1efdcbeb44e94a3bb3acb45daa0e7197" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 45203 } } peers { permanent_uuid: "caf849fdfdc048128ec5b5502845312a" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 41443 } }
I20260811 18:57:46.513720 23099 raft_consensus.cc:729] T d0efa96143d74ee8b986a5143df2e038 P caf849fdfdc048128ec5b5502845312a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: caf849fdfdc048128ec5b5502845312a, State: Initialized, Role: FOLLOWER
I20260811 18:57:46.513863 22946 ts_manager.cc:194] Registered new tserver with Master: caf849fdfdc048128ec5b5502845312a (127.27.180.129:41443)
I20260811 18:57:46.514166 23099 consensus_queue.cc:260] T d0efa96143d74ee8b986a5143df2e038 P caf849fdfdc048128ec5b5502845312a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d37ed58d1e4e4273a7ad3a2a20681c90" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 36347 } } peers { permanent_uuid: "1efdcbeb44e94a3bb3acb45daa0e7197" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 45203 } } peers { permanent_uuid: "caf849fdfdc048128ec5b5502845312a" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 41443 } }
I20260811 18:57:46.514935 23099 ts_tablet_manager.cc:1428] T d0efa96143d74ee8b986a5143df2e038 P caf849fdfdc048128ec5b5502845312a: Time spent starting tablet: real 0.004s user 0.005s sys 0.001s
I20260811 18:57:46.517827 22946 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.180.129@KRBTEST.COM'} at 127.27.180.129:51693
I20260811 18:57:46.520304 23277 heartbeater.cc:503] Master 127.27.180.190:40989 was elected leader, sending a full tablet report...
Aug 11 18:57:46 dist-test-slave-2j7r krb5kdc[21860](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474666, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.190@KRBTEST.COM for kudu/127.27.180.129@KRBTEST.COM
I20260811 18:57:46.527518 23289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:46.517771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38786 (local address 127.27.180.129:41443)
0811 18:57:46.518049 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:46.518060 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:46.518080 (+ 20us) server_negotiation.cc:407] Connection header received
0811 18:57:46.518143 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:46.518147 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:46.518337 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:46.518454 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:46.519233 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:46.520335 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:46.521408 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:46.521733 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:46.521830 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:46.524187 (+ 2357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:46.524196 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:46.524205 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:46.524228 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:46.525697 (+ 1469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:46.526213 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:46.526217 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:46.526218 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:46.526324 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:46.526585 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:46.526587 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:46.526588 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:46.526802 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:46.526900 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:46.527263 (+ 363us) server_negotiation.cc:299] Negotiation successful
0811 18:57:46.527396 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":95,"threads_started":1}
I20260811 18:57:46.527545 23288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:46.517128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.129:41443 (local address 127.0.0.1:38786)
0811 18:57:46.517491 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:46.517533 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:46.517649 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:46.518484 (+ 835us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:46.518490 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:46.518904 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:57:46.519062 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:46.519071 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:46.520499 (+ 1428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:46.520503 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:46.521288 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:46.521298 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:46.521851 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:46.521855 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:46.521963 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:46.522464 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:57:46.522479 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:57:46.524052 (+ 1573us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:57:46.525803 (+ 1751us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:46.525813 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:46.525818 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:46.526116 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:46.526408 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:46.526412 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:46.526414 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:46.526507 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:46.526903 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:57:46.526909 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:57:46.527035 (+ 126us) client_negotiation.cc:769] Sending connection context
0811 18:57:46.527185 (+ 150us) client_negotiation.cc:240] Negotiation successful
0811 18:57:46.527376 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":102,"threads_started":1}
I20260811 18:57:46.528482 23176 tablet_service.cc:1514] Processing DeleteTablet for tablet d0efa96143d74ee8b986a5143df2e038 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:46 UTC) from {username='kudu', principal='kudu/127.27.180.190@KRBTEST.COM'} at 127.0.0.1:38786
I20260811 18:57:46.529320 23290 tablet_replica.cc:330] stopping tablet replica
I20260811 18:57:46.529660 23290 raft_consensus.cc:2229] T d0efa96143d74ee8b986a5143df2e038 P caf849fdfdc048128ec5b5502845312a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:46.529925 23290 raft_consensus.cc:2258] T d0efa96143d74ee8b986a5143df2e038 P caf849fdfdc048128ec5b5502845312a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:46.531113 23290 ts_tablet_manager.cc:1905] T d0efa96143d74ee8b986a5143df2e038 P caf849fdfdc048128ec5b5502845312a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:46.535645 23290 ts_tablet_manager.cc:1918] T d0efa96143d74ee8b986a5143df2e038 P caf849fdfdc048128ec5b5502845312a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:57:46.535758 23290 log.cc:1196] T d0efa96143d74ee8b986a5143df2e038 P caf849fdfdc048128ec5b5502845312a: Deleting WAL directory at /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-0/wal/wals/d0efa96143d74ee8b986a5143df2e038
I20260811 18:57:46.536031 23290 ts_tablet_manager.cc:1939] T d0efa96143d74ee8b986a5143df2e038 P caf849fdfdc048128ec5b5502845312a: Deleting consensus metadata
I20260811 18:57:46.537956 22914 catalog_manager.cc:4841] TS caf849fdfdc048128ec5b5502845312a (127.27.180.129:41443): tablet d0efa96143d74ee8b986a5143df2e038 (table test-table [id=550be8bb25cf4e20939aae1a59d83bce]) successfully deleted
W20260811 18:57:46.616745 23282 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:57:46.616986 23282 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:57:46.617026 23282 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:57:46.620962 23282 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:57:46.621018 23282 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:57:46.621074 23282 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.180.130
I20260811 18:57:46.626186 23282 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:42327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.130.keytab
--principal=kudu/127.27.180.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.180.130:36347
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.180.130
--webserver_port=40669
--tserver_master_addrs=127.27.180.190:40989
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23282
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.180.130
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:57:46.627147 23282 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:57:46.628106 23282 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:57:46.637496 23304 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:46.637554 23306 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:57:46.637547 23282 server_base.cc:1034] running on GCE node
W20260811 18:57:46.637522 23310 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:57:46.638120 23282 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:57:46.638718 23282 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:57:46.639884 23282 hybrid_clock.cc:648] HybridClock initialized: now 1786474666639869 us; error 28 us; skew 500 ppm
Aug 11 18:57:46 dist-test-slave-2j7r krb5kdc[21860](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474666, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:57:46.642594 23282 init.cc:377] Logged in from keytab as kudu/127.27.180.130@KRBTEST.COM (short username kudu)
I20260811 18:57:46.643407 23282 webserver.cc:460] Webserver started at http://127.27.180.130:40669/ using document root <none> and password file <none>
I20260811 18:57:46.643944 23282 fs_manager.cc:359] Metadata directory not provided
I20260811 18:57:46.644026 23282 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:57:46.647967 23282 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260811 18:57:46.650044 23321 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:46.651204 23282 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 18:57:46.651314 23282 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-1/wal
uuid: "d37ed58d1e4e4273a7ad3a2a20681c90"
format_stamp: "Formatted at 2026-08-11 18:57:45 on dist-test-slave-2j7r"
server_key: "15916bcb143779610fea979bb97c7f57"
server_key_iv: "e234587a7b46cce4a2a22c70a0ec694d"
server_key_version: "encryptionkey@0"
I20260811 18:57:46.651842 23282 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:57:46.662658 23282 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:57:46.665303 23282 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:57:46.665468 23282 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:57:46.666039 23282 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:57:46.666841 23282 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:57:46.666888 23282 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:46.666940 23282 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:57:46.666970 23282 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:46.675114 23282 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.130:36347
I20260811 18:57:46.675143 23531 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.130:36347 every 8 connection(s)
I20260811 18:57:46.676040 23282 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-1/data/info.pb
I20260811 18:57:46.682436 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 23282
I20260811 18:57:46.682843 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.180.131:45203
--local_ip_for_outbound_sockets=127.27.180.131
--tserver_master_addrs=127.27.180.190:40989
--webserver_port=46285
--webserver_interface=127.27.180.131
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.131.keytab
--principal=kudu/127.27.180.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:42327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 18:57:46 dist-test-slave-2j7r krb5kdc[21860](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474666, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.130@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM
I20260811 18:57:46.689219 23279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:46.677750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.130:48931 (local address 127.27.180.190:40989)
0811 18:57:46.677888 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:46.677891 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:46.678664 (+ 773us) server_negotiation.cc:407] Connection header received
0811 18:57:46.680014 (+ 1350us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:46.680018 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:46.680083 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:46.680154 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:46.681328 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:46.681858 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:46.683018 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:46.683272 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:46.683333 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:46.685965 (+ 2632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:46.685969 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:46.685971 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:46.685996 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:46.687224 (+ 1228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:46.687706 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:46.687709 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:46.687710 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:46.687795 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:46.688110 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:46.688114 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:46.688115 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:46.688302 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:46.688393 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:46.688875 (+ 482us) server_negotiation.cc:299] Negotiation successful
0811 18:57:46.689018 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 18:57:46.690024 23539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:46.678099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:40989 (local address 127.27.180.130:48931)
0811 18:57:46.678526 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:46.678566 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:46.679810 (+ 1244us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:46.680343 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:46.680350 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:46.680682 (+ 332us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:57:46.681181 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:46.681198 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:46.681976 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:46.681978 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:46.682881 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:46.682892 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:46.683507 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:46.683510 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:46.683657 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:46.684198 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:57:46.684214 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:57:46.685788 (+ 1574us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:57:46.687339 (+ 1551us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:46.687345 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:46.687361 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:46.687610 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:46.687879 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:46.687882 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:46.687884 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:46.688020 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:46.688403 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:57:46.688410 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:57:46.688649 (+ 239us) client_negotiation.cc:769] Sending connection context
0811 18:57:46.688818 (+ 169us) client_negotiation.cc:240] Negotiation successful
0811 18:57:46.689176 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":121,"threads_started":1}
I20260811 18:57:46.691183 23532 heartbeater.cc:347] Connected to a master server at 127.27.180.190:40989
I20260811 18:57:46.691417 23532 heartbeater.cc:464] Registering TS with master...
I20260811 18:57:46.691906 23532 heartbeater.cc:511] Master 127.27.180.190:40989 requested a full tablet report, sending...
I20260811 18:57:46.692974 22946 ts_manager.cc:194] Registered new tserver with Master: d37ed58d1e4e4273a7ad3a2a20681c90 (127.27.180.130:36347)
I20260811 18:57:46.693820 22946 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.180.130@KRBTEST.COM'} at 127.27.180.130:48931
W20260811 18:57:46.796595 23541 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:57:46.796834 23541 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:57:46.796875 23541 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:57:46.800776 23541 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:57:46.800827 23541 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:57:46.800882 23541 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.180.131
I20260811 18:57:46.805985 23541 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:42327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.131.keytab
--principal=kudu/127.27.180.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.180.131:45203
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.180.131
--webserver_port=46285
--tserver_master_addrs=127.27.180.190:40989
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23541
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.180.131
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:57:46.806944 23541 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:57:46.807883 23541 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:57:46.817009 23556 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:57:46.817207 23541 server_base.cc:1034] running on GCE node
W20260811 18:57:46.817238 23560 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:46.817040 23557 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:57:46.817689 23541 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:57:46.818281 23541 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:57:46.819435 23541 hybrid_clock.cc:648] HybridClock initialized: now 1786474666819425 us; error 30 us; skew 500 ppm
Aug 11 18:57:46 dist-test-slave-2j7r krb5kdc[21860](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474666, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:57:46.822057 23541 init.cc:377] Logged in from keytab as kudu/127.27.180.131@KRBTEST.COM (short username kudu)
I20260811 18:57:46.822856 23541 webserver.cc:460] Webserver started at http://127.27.180.131:46285/ using document root <none> and password file <none>
I20260811 18:57:46.823359 23541 fs_manager.cc:359] Metadata directory not provided
I20260811 18:57:46.823431 23541 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:57:46.827064 23541 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.002s
I20260811 18:57:46.828949 23576 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:46.830163 23541 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 18:57:46.830259 23541 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-2/wal
uuid: "1efdcbeb44e94a3bb3acb45daa0e7197"
format_stamp: "Formatted at 2026-08-11 18:57:45 on dist-test-slave-2j7r"
server_key: "702b7bf9382ecdd0b10b3c3fd01da645"
server_key_iv: "5326459e8b34be2c10b5b59c40bac440"
server_key_version: "encryptionkey@0"
I20260811 18:57:46.830809 23541 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:57:46.853778 23541 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:57:46.856817 23541 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:57:46.856964 23541 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:57:46.857540 23541 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:57:46.858511 23541 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:57:46.858589 23541 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:46.858639 23541 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:57:46.858664 23541 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:46.868712 23541 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.131:45203
I20260811 18:57:46.868734 23794 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.131:45203 every 8 connection(s)
I20260811 18:57:46.869971 23541 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-2/data/info.pb
I20260811 18:57:46.870465 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 23541
I20260811 18:57:46.879071 23279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:46.871629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47728 (local address 127.27.180.190:40989)
0811 18:57:46.871752 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:46.871755 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:46.871770 (+ 15us) server_negotiation.cc:407] Connection header received
0811 18:57:46.871812 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:46.871814 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:46.871855 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:46.871922 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:46.872849 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:46.873471 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:46.874346 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:46.874567 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:46.874655 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:46.876043 (+ 1388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:46.876048 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:46.876050 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:46.876079 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:46.877482 (+ 1403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:46.877870 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:46.877873 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:46.877875 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:46.877964 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:46.878220 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:46.878223 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:46.878225 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:46.878460 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:46.878646 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:46.878798 (+ 152us) server_negotiation.cc:299] Negotiation successful
0811 18:57:46.878909 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":41}
Aug 11 18:57:46 dist-test-slave-2j7r krb5kdc[21860](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474666, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.131@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM
I20260811 18:57:46.884013 23801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:46.872158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.131:45959 (local address 127.27.180.190:40989)
0811 18:57:46.872956 (+ 798us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:46.872961 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:46.873445 (+ 484us) server_negotiation.cc:407] Connection header received
0811 18:57:46.874795 (+ 1350us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:46.874798 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:46.874844 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:46.874931 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:57:46.876062 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:46.876743 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:46.877978 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:46.878277 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:46.878367 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:46.880998 (+ 2631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:57:46.881002 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:57:46.881004 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:57:46.881029 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:57:46.882179 (+ 1150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:46.882629 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:46.882631 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:46.882632 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:46.882699 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:57:46.882963 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:57:46.882965 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:57:46.882966 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:57:46.883113 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:57:46.883183 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:46.883756 (+ 573us) server_negotiation.cc:299] Negotiation successful
0811 18:57:46.883884 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":726,"spinlock_wait_cycles":24448,"thread_start_us":60,"threads_started":1}
I20260811 18:57:46.884737 23804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:46.872793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:40989 (local address 127.27.180.131:45959)
0811 18:57:46.873298 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:46.873349 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:46.874624 (+ 1275us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:46.875079 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:46.875087 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:46.875494 (+ 407us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:57:46.875949 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:46.875963 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:46.876909 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:46.876913 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:46.877861 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:46.877872 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:46.878422 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:46.878425 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:46.878565 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:46.879046 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:57:46.879063 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:57:46.880825 (+ 1762us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:57:46.882297 (+ 1472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:46.882302 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:46.882313 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:46.882551 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:46.882777 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:57:46.882779 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:57:46.882780 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:57:46.882899 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:57:46.883255 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:57:46.883260 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:57:46.883493 (+ 233us) client_negotiation.cc:769] Sending connection context
0811 18:57:46.883660 (+ 167us) client_negotiation.cc:240] Negotiation successful
0811 18:57:46.883928 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":128,"threads_started":1}
I20260811 18:57:46.885797 23795 heartbeater.cc:347] Connected to a master server at 127.27.180.190:40989
I20260811 18:57:46.886061 23795 heartbeater.cc:464] Registering TS with master...
I20260811 18:57:46.886556 23795 heartbeater.cc:511] Master 127.27.180.190:40989 requested a full tablet report, sending...
I20260811 18:57:46.887589 22946 ts_manager.cc:194] Registered new tserver with Master: 1efdcbeb44e94a3bb3acb45daa0e7197 (127.27.180.131:45203)
I20260811 18:57:46.888140 22946 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.180.131@KRBTEST.COM'} at 127.27.180.131:45959
I20260811 18:57:46.892863 28370 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 18:57:46.897024 23801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:46.893514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47736 (local address 127.27.180.190:40989)
0811 18:57:46.893647 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:46.893652 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:46.893829 (+ 177us) server_negotiation.cc:407] Connection header received
0811 18:57:46.894001 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:46.894005 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:46.894098 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:46.894206 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:57:46.894429 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:46.894958 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:46.895875 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:46.896102 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:46.896183 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:46.896398 (+ 215us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:57:46.896736 (+ 338us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:57:46.896827 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:46.896853 (+ 26us) server_negotiation.cc:299] Negotiation successful
0811 18:57:46.896900 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 18:57:46.897615 22946 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:47736:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:57:46.900740 22946 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:57:46.909188 23176 tablet_service.cc:1467] Processing CreateTablet for tablet 8edc1d0ef4524b6087a0d283537ac75f (DEFAULT_TABLE table=test-table [id=a7f1d442a7124aeb8b69d6c9422884bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:46.909884 23176 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8edc1d0ef4524b6087a0d283537ac75f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:46.912597 23823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:46.907400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.130:36347 (local address 127.0.0.1:47342)
0811 18:57:46.908136 (+ 736us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:46.908151 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:46.908250 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:46.908749 (+ 499us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:46.908751 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:46.908769 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:57:46.908866 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:46.908873 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:46.910143 (+ 1270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:46.910146 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:46.910945 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:46.910954 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:46.912182 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:46.912186 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:46.912280 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:46.912327 (+ 47us) client_negotiation.cc:769] Sending connection context
0811 18:57:46.912399 (+ 72us) client_negotiation.cc:240] Negotiation successful
0811 18:57:46.912472 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":656}
I20260811 18:57:46.913009 23821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:46.907614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47342 (local address 127.27.180.130:36347)
0811 18:57:46.908026 (+ 412us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:46.908038 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:46.908351 (+ 313us) server_negotiation.cc:407] Connection header received
0811 18:57:46.908441 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:46.908447 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:46.908613 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:46.908755 (+ 142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:57:46.909101 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:46.910009 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:46.911065 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:46.911880 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:46.911995 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:46.912117 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:46.912765 (+ 648us) server_negotiation.cc:299] Negotiation successful
0811 18:57:46.912873 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":248,"thread_start_us":112,"threads_started":1}
I20260811 18:57:46.913038 23099 tablet_bootstrap.cc:492] T 8edc1d0ef4524b6087a0d283537ac75f P caf849fdfdc048128ec5b5502845312a: Bootstrap starting.
I20260811 18:57:46.913033 23288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:46.907838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.131:45203 (local address 127.0.0.1:34488)
0811 18:57:46.908260 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:46.908273 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:46.908341 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:46.909389 (+ 1048us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:46.909392 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:46.909412 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:57:46.909496 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:46.909502 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:46.910918 (+ 1416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:46.910922 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:46.911722 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:46.911733 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:46.912647 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:46.912651 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:46.912746 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:46.912770 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 18:57:46.912827 (+ 57us) client_negotiation.cc:240] Negotiation successful
0811 18:57:46.912902 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":101,"threads_started":1}
I20260811 18:57:46.913806 23099 tablet_bootstrap.cc:654] T 8edc1d0ef4524b6087a0d283537ac75f P caf849fdfdc048128ec5b5502845312a: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:46.913194 23826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:46.908418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34488 (local address 127.27.180.131:45203)
0811 18:57:46.908752 (+ 334us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:46.908763 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:46.908794 (+ 31us) server_negotiation.cc:407] Connection header received
0811 18:57:46.908865 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:46.908870 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:46.909230 (+ 360us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:46.909641 (+ 411us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:57:46.909797 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:46.910776 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:46.911863 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:46.912531 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:46.912646 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:46.912903 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:46.912951 (+ 48us) server_negotiation.cc:299] Negotiation successful
0811 18:57:46.913037 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":245,"thread_start_us":133,"threads_started":1}
I20260811 18:57:46.915688 23099 tablet_bootstrap.cc:492] T 8edc1d0ef4524b6087a0d283537ac75f P caf849fdfdc048128ec5b5502845312a: No bootstrap required, opened a new log
I20260811 18:57:46.915778 23099 ts_tablet_manager.cc:1397] T 8edc1d0ef4524b6087a0d283537ac75f P caf849fdfdc048128ec5b5502845312a: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 18:57:46.916232 23099 raft_consensus.cc:348] T 8edc1d0ef4524b6087a0d283537ac75f P caf849fdfdc048128ec5b5502845312a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d37ed58d1e4e4273a7ad3a2a20681c90" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 36347 } } peers { permanent_uuid: "1efdcbeb44e94a3bb3acb45daa0e7197" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 45203 } } peers { permanent_uuid: "caf849fdfdc048128ec5b5502845312a" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 41443 } }
I20260811 18:57:46.916349 23099 raft_consensus.cc:374] T 8edc1d0ef4524b6087a0d283537ac75f P caf849fdfdc048128ec5b5502845312a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:46.916385 23099 raft_consensus.cc:729] T 8edc1d0ef4524b6087a0d283537ac75f P caf849fdfdc048128ec5b5502845312a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: caf849fdfdc048128ec5b5502845312a, State: Initialized, Role: FOLLOWER
I20260811 18:57:46.916049 23407 tablet_service.cc:1467] Processing CreateTablet for tablet 8edc1d0ef4524b6087a0d283537ac75f (DEFAULT_TABLE table=test-table [id=a7f1d442a7124aeb8b69d6c9422884bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:46.916504 23099 consensus_queue.cc:260] T 8edc1d0ef4524b6087a0d283537ac75f P caf849fdfdc048128ec5b5502845312a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d37ed58d1e4e4273a7ad3a2a20681c90" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 36347 } } peers { permanent_uuid: "1efdcbeb44e94a3bb3acb45daa0e7197" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 45203 } } peers { permanent_uuid: "caf849fdfdc048128ec5b5502845312a" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 41443 } }
I20260811 18:57:46.916765 23099 ts_tablet_manager.cc:1428] T 8edc1d0ef4524b6087a0d283537ac75f P caf849fdfdc048128ec5b5502845312a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 18:57:46.916483 23672 tablet_service.cc:1467] Processing CreateTablet for tablet 8edc1d0ef4524b6087a0d283537ac75f (DEFAULT_TABLE table=test-table [id=a7f1d442a7124aeb8b69d6c9422884bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:57:46.917199 23407 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8edc1d0ef4524b6087a0d283537ac75f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:46.917514 23672 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8edc1d0ef4524b6087a0d283537ac75f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:57:46.923974 23829 tablet_bootstrap.cc:492] T 8edc1d0ef4524b6087a0d283537ac75f P 1efdcbeb44e94a3bb3acb45daa0e7197: Bootstrap starting.
I20260811 18:57:46.923978 23830 tablet_bootstrap.cc:492] T 8edc1d0ef4524b6087a0d283537ac75f P d37ed58d1e4e4273a7ad3a2a20681c90: Bootstrap starting.
I20260811 18:57:46.926322 23829 tablet_bootstrap.cc:654] T 8edc1d0ef4524b6087a0d283537ac75f P 1efdcbeb44e94a3bb3acb45daa0e7197: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:46.926983 23830 tablet_bootstrap.cc:654] T 8edc1d0ef4524b6087a0d283537ac75f P d37ed58d1e4e4273a7ad3a2a20681c90: Neither blocks nor log segments found. Creating new log.
I20260811 18:57:46.927057 23829 log.cc:824] T 8edc1d0ef4524b6087a0d283537ac75f P 1efdcbeb44e94a3bb3acb45daa0e7197: Log is configured to *not* fsync() on all Append() calls
I20260811 18:57:46.927672 23830 log.cc:824] T 8edc1d0ef4524b6087a0d283537ac75f P d37ed58d1e4e4273a7ad3a2a20681c90: Log is configured to *not* fsync() on all Append() calls
I20260811 18:57:46.928802 23829 tablet_bootstrap.cc:492] T 8edc1d0ef4524b6087a0d283537ac75f P 1efdcbeb44e94a3bb3acb45daa0e7197: No bootstrap required, opened a new log
I20260811 18:57:46.928979 23829 ts_tablet_manager.cc:1397] T 8edc1d0ef4524b6087a0d283537ac75f P 1efdcbeb44e94a3bb3acb45daa0e7197: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 18:57:46.929368 23830 tablet_bootstrap.cc:492] T 8edc1d0ef4524b6087a0d283537ac75f P d37ed58d1e4e4273a7ad3a2a20681c90: No bootstrap required, opened a new log
I20260811 18:57:46.929538 23830 ts_tablet_manager.cc:1397] T 8edc1d0ef4524b6087a0d283537ac75f P d37ed58d1e4e4273a7ad3a2a20681c90: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 18:57:46.932137 23829 raft_consensus.cc:348] T 8edc1d0ef4524b6087a0d283537ac75f P 1efdcbeb44e94a3bb3acb45daa0e7197 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d37ed58d1e4e4273a7ad3a2a20681c90" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 36347 } } peers { permanent_uuid: "1efdcbeb44e94a3bb3acb45daa0e7197" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 45203 } } peers { permanent_uuid: "caf849fdfdc048128ec5b5502845312a" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 41443 } }
I20260811 18:57:46.932355 23829 raft_consensus.cc:374] T 8edc1d0ef4524b6087a0d283537ac75f P 1efdcbeb44e94a3bb3acb45daa0e7197 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:46.932392 23829 raft_consensus.cc:729] T 8edc1d0ef4524b6087a0d283537ac75f P 1efdcbeb44e94a3bb3acb45daa0e7197 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1efdcbeb44e94a3bb3acb45daa0e7197, State: Initialized, Role: FOLLOWER
I20260811 18:57:46.932687 23830 raft_consensus.cc:348] T 8edc1d0ef4524b6087a0d283537ac75f P d37ed58d1e4e4273a7ad3a2a20681c90 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d37ed58d1e4e4273a7ad3a2a20681c90" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 36347 } } peers { permanent_uuid: "1efdcbeb44e94a3bb3acb45daa0e7197" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 45203 } } peers { permanent_uuid: "caf849fdfdc048128ec5b5502845312a" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 41443 } }
I20260811 18:57:46.932790 23829 consensus_queue.cc:260] T 8edc1d0ef4524b6087a0d283537ac75f P 1efdcbeb44e94a3bb3acb45daa0e7197 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d37ed58d1e4e4273a7ad3a2a20681c90" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 36347 } } peers { permanent_uuid: "1efdcbeb44e94a3bb3acb45daa0e7197" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 45203 } } peers { permanent_uuid: "caf849fdfdc048128ec5b5502845312a" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 41443 } }
I20260811 18:57:46.932893 23830 raft_consensus.cc:374] T 8edc1d0ef4524b6087a0d283537ac75f P d37ed58d1e4e4273a7ad3a2a20681c90 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:57:46.932960 23830 raft_consensus.cc:729] T 8edc1d0ef4524b6087a0d283537ac75f P d37ed58d1e4e4273a7ad3a2a20681c90 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d37ed58d1e4e4273a7ad3a2a20681c90, State: Initialized, Role: FOLLOWER
I20260811 18:57:46.933400 23830 consensus_queue.cc:260] T 8edc1d0ef4524b6087a0d283537ac75f P d37ed58d1e4e4273a7ad3a2a20681c90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d37ed58d1e4e4273a7ad3a2a20681c90" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 36347 } } peers { permanent_uuid: "1efdcbeb44e94a3bb3acb45daa0e7197" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 45203 } } peers { permanent_uuid: "caf849fdfdc048128ec5b5502845312a" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 41443 } }
I20260811 18:57:46.933526 23795 heartbeater.cc:503] Master 127.27.180.190:40989 was elected leader, sending a full tablet report...
I20260811 18:57:46.933740 23829 ts_tablet_manager.cc:1428] T 8edc1d0ef4524b6087a0d283537ac75f P 1efdcbeb44e94a3bb3acb45daa0e7197: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260811 18:57:46.934237 23532 heartbeater.cc:503] Master 127.27.180.190:40989 was elected leader, sending a full tablet report...
I20260811 18:57:46.934446 23830 ts_tablet_manager.cc:1428] T 8edc1d0ef4524b6087a0d283537ac75f P d37ed58d1e4e4273a7ad3a2a20681c90: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260811 18:57:46.950675 23838 raft_consensus.cc:482] T 8edc1d0ef4524b6087a0d283537ac75f P d37ed58d1e4e4273a7ad3a2a20681c90 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:57:46.950879 23838 raft_consensus.cc:504] T 8edc1d0ef4524b6087a0d283537ac75f P d37ed58d1e4e4273a7ad3a2a20681c90 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d37ed58d1e4e4273a7ad3a2a20681c90" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 36347 } } peers { permanent_uuid: "1efdcbeb44e94a3bb3acb45daa0e7197" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 45203 } } peers { permanent_uuid: "caf849fdfdc048128ec5b5502845312a" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 41443 } }
I20260811 18:57:46.952121 23838 leader_election.cc:288] T 8edc1d0ef4524b6087a0d283537ac75f P d37ed58d1e4e4273a7ad3a2a20681c90 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1efdcbeb44e94a3bb3acb45daa0e7197 (127.27.180.131:45203), caf849fdfdc048128ec5b5502845312a (127.27.180.129:41443)
I20260811 18:57:46.956367 23539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:46.952342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.131:45203 (local address 127.27.180.130:49177)
0811 18:57:46.952534 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:46.952551 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:46.952647 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:46.952939 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:46.952942 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:46.952963 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:57:46.953162 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:46.953182 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:46.954014 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:46.954017 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:46.955099 (+ 1082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:46.955109 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:46.955935 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:46.955939 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:46.956045 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:46.956071 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 18:57:46.956138 (+ 67us) client_negotiation.cc:240] Negotiation successful
0811 18:57:46.956213 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"mutex_wait_us":25}
I20260811 18:57:46.956600 23839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:46.952363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.129:41443 (local address 127.27.180.130:54429)
0811 18:57:46.952811 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:46.952824 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:46.952920 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:46.953279 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:57:46.953282 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:57:46.953303 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:57:46.953419 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:46.953426 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:46.954393 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:46.954395 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:46.955333 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:57:46.955344 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:57:46.956188 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:57:46.956192 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:57:46.956299 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:46.956324 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 18:57:46.956391 (+ 67us) client_negotiation.cc:240] Negotiation successful
0811 18:57:46.956457 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":376,"mutex_wait_us":23,"thread_start_us":116,"threads_started":1}
I20260811 18:57:46.956609 23289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:46.952719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.130:54429 (local address 127.27.180.129:41443)
0811 18:57:46.952852 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:46.952855 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:46.952883 (+ 28us) server_negotiation.cc:407] Connection header received
0811 18:57:46.953021 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:46.953025 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:46.953130 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:46.953247 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:57:46.953574 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:46.954280 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:46.955458 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:46.956055 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:46.956169 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:46.956320 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:46.956424 (+ 104us) server_negotiation.cc:299] Negotiation successful
0811 18:57:46.956497 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 18:57:46.956902 23826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:46.952478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.130:49177 (local address 127.27.180.131:45203)
0811 18:57:46.952617 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:46.952622 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:46.952639 (+ 17us) server_negotiation.cc:407] Connection header received
0811 18:57:46.952763 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:46.952766 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:46.952818 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:46.952908 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:57:46.953310 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:46.953910 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:46.955219 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:46.955811 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:46.955932 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:46.955986 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:46.956695 (+ 709us) server_negotiation.cc:299] Negotiation successful
0811 18:57:46.956777 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 18:57:46.957290 23209 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8edc1d0ef4524b6087a0d283537ac75f" candidate_uuid: "d37ed58d1e4e4273a7ad3a2a20681c90" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "caf849fdfdc048128ec5b5502845312a" is_pre_election: true
I20260811 18:57:46.957440 23714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8edc1d0ef4524b6087a0d283537ac75f" candidate_uuid: "d37ed58d1e4e4273a7ad3a2a20681c90" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efdcbeb44e94a3bb3acb45daa0e7197" is_pre_election: true
I20260811 18:57:46.957584 23209 raft_consensus.cc:2454] T 8edc1d0ef4524b6087a0d283537ac75f P caf849fdfdc048128ec5b5502845312a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d37ed58d1e4e4273a7ad3a2a20681c90 in term 0.
I20260811 18:57:46.957734 23714 raft_consensus.cc:2454] T 8edc1d0ef4524b6087a0d283537ac75f P 1efdcbeb44e94a3bb3acb45daa0e7197 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d37ed58d1e4e4273a7ad3a2a20681c90 in term 0.
I20260811 18:57:46.958032 23322 leader_election.cc:302] T 8edc1d0ef4524b6087a0d283537ac75f P d37ed58d1e4e4273a7ad3a2a20681c90 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: caf849fdfdc048128ec5b5502845312a, d37ed58d1e4e4273a7ad3a2a20681c90; no voters:
I20260811 18:57:46.958302 23838 raft_consensus.cc:2781] T 8edc1d0ef4524b6087a0d283537ac75f P d37ed58d1e4e4273a7ad3a2a20681c90 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:57:46.958392 23838 raft_consensus.cc:482] T 8edc1d0ef4524b6087a0d283537ac75f P d37ed58d1e4e4273a7ad3a2a20681c90 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:57:46.958443 23838 raft_consensus.cc:3037] T 8edc1d0ef4524b6087a0d283537ac75f P d37ed58d1e4e4273a7ad3a2a20681c90 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:46.959460 23838 raft_consensus.cc:504] T 8edc1d0ef4524b6087a0d283537ac75f P d37ed58d1e4e4273a7ad3a2a20681c90 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d37ed58d1e4e4273a7ad3a2a20681c90" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 36347 } } peers { permanent_uuid: "1efdcbeb44e94a3bb3acb45daa0e7197" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 45203 } } peers { permanent_uuid: "caf849fdfdc048128ec5b5502845312a" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 41443 } }
I20260811 18:57:46.959760 23838 leader_election.cc:288] T 8edc1d0ef4524b6087a0d283537ac75f P d37ed58d1e4e4273a7ad3a2a20681c90 [CANDIDATE]: Term 1 election: Requested vote from peers 1efdcbeb44e94a3bb3acb45daa0e7197 (127.27.180.131:45203), caf849fdfdc048128ec5b5502845312a (127.27.180.129:41443)
I20260811 18:57:46.960083 23714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8edc1d0ef4524b6087a0d283537ac75f" candidate_uuid: "d37ed58d1e4e4273a7ad3a2a20681c90" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1efdcbeb44e94a3bb3acb45daa0e7197"
I20260811 18:57:46.960094 23209 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8edc1d0ef4524b6087a0d283537ac75f" candidate_uuid: "d37ed58d1e4e4273a7ad3a2a20681c90" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "caf849fdfdc048128ec5b5502845312a"
I20260811 18:57:46.960223 23209 raft_consensus.cc:3037] T 8edc1d0ef4524b6087a0d283537ac75f P caf849fdfdc048128ec5b5502845312a [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:46.960210 23714 raft_consensus.cc:3037] T 8edc1d0ef4524b6087a0d283537ac75f P 1efdcbeb44e94a3bb3acb45daa0e7197 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:57:46.961534 23209 raft_consensus.cc:2454] T 8edc1d0ef4524b6087a0d283537ac75f P caf849fdfdc048128ec5b5502845312a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d37ed58d1e4e4273a7ad3a2a20681c90 in term 1.
I20260811 18:57:46.961534 23714 raft_consensus.cc:2454] T 8edc1d0ef4524b6087a0d283537ac75f P 1efdcbeb44e94a3bb3acb45daa0e7197 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d37ed58d1e4e4273a7ad3a2a20681c90 in term 1.
I20260811 18:57:46.961798 23322 leader_election.cc:302] T 8edc1d0ef4524b6087a0d283537ac75f P d37ed58d1e4e4273a7ad3a2a20681c90 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: caf849fdfdc048128ec5b5502845312a, d37ed58d1e4e4273a7ad3a2a20681c90; no voters:
I20260811 18:57:46.961962 23838 raft_consensus.cc:2781] T 8edc1d0ef4524b6087a0d283537ac75f P d37ed58d1e4e4273a7ad3a2a20681c90 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:57:46.962212 23838 raft_consensus.cc:686] T 8edc1d0ef4524b6087a0d283537ac75f P d37ed58d1e4e4273a7ad3a2a20681c90 [term 1 LEADER]: Becoming Leader. State: Replica: d37ed58d1e4e4273a7ad3a2a20681c90, State: Running, Role: LEADER
I20260811 18:57:46.962538 23838 consensus_queue.cc:237] T 8edc1d0ef4524b6087a0d283537ac75f P d37ed58d1e4e4273a7ad3a2a20681c90 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d37ed58d1e4e4273a7ad3a2a20681c90" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 36347 } } peers { permanent_uuid: "1efdcbeb44e94a3bb3acb45daa0e7197" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 45203 } } peers { permanent_uuid: "caf849fdfdc048128ec5b5502845312a" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 41443 } }
I20260811 18:57:46.965636 22946 catalog_manager.cc:5495] T 8edc1d0ef4524b6087a0d283537ac75f P d37ed58d1e4e4273a7ad3a2a20681c90 reported cstate change: term changed from 0 to 1, leader changed from <none> to d37ed58d1e4e4273a7ad3a2a20681c90 (127.27.180.130). New cstate: current_term: 1 leader_uuid: "d37ed58d1e4e4273a7ad3a2a20681c90" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d37ed58d1e4e4273a7ad3a2a20681c90" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 36347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1efdcbeb44e94a3bb3acb45daa0e7197" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 45203 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "caf849fdfdc048128ec5b5502845312a" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 41443 } health_report { overall_health: UNKNOWN } } }
I20260811 18:57:46.976274 23821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:46.972952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47352 (local address 127.27.180.130:36347)
0811 18:57:46.973077 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0811 18:57:46.973102 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0811 18:57:46.973119 (+ 17us) server_negotiation.cc:407] Connection header received
0811 18:57:46.973246 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:57:46.973249 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:57:46.973303 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:57:46.973386 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:57:46.973622 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:46.974122 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:46.974948 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:57:46.975160 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:57:46.975240 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:57:46.975550 (+ 310us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:57:46.975953 (+ 403us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:57:46.976040 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 18:57:46.976084 (+ 44us) server_negotiation.cc:299] Negotiation successful
0811 18:57:46.976134 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 18:57:46.982617 23209 raft_consensus.cc:1261] T 8edc1d0ef4524b6087a0d283537ac75f P caf849fdfdc048128ec5b5502845312a [term 1 FOLLOWER]: Refusing update from remote peer d37ed58d1e4e4273a7ad3a2a20681c90: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:57:46.982620 23714 raft_consensus.cc:1261] T 8edc1d0ef4524b6087a0d283537ac75f P 1efdcbeb44e94a3bb3acb45daa0e7197 [term 1 FOLLOWER]: Refusing update from remote peer d37ed58d1e4e4273a7ad3a2a20681c90: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:57:46.983464 23840 consensus_queue.cc:1035] T 8edc1d0ef4524b6087a0d283537ac75f P d37ed58d1e4e4273a7ad3a2a20681c90 [LEADER]: Connected to new peer: Peer: permanent_uuid: "caf849fdfdc048128ec5b5502845312a" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 41443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:46.983669 23838 consensus_queue.cc:1035] T 8edc1d0ef4524b6087a0d283537ac75f P d37ed58d1e4e4273a7ad3a2a20681c90 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1efdcbeb44e94a3bb3acb45daa0e7197" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 45203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:57:46.999117 22946 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:47736:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:57:46.999317 22946 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:47736:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:57:47.001816 22946 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 46e6feef897141feb6bf6aba7a68a7ce: Sending DeleteTablet for 3 replicas of tablet 8edc1d0ef4524b6087a0d283537ac75f
I20260811 18:57:47.002540 23176 tablet_service.cc:1514] Processing DeleteTablet for tablet 8edc1d0ef4524b6087a0d283537ac75f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:46 UTC) from {username='kudu', principal='kudu/127.27.180.190@KRBTEST.COM'} at 127.0.0.1:38786
I20260811 18:57:47.002622 23672 tablet_service.cc:1514] Processing DeleteTablet for tablet 8edc1d0ef4524b6087a0d283537ac75f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:46 UTC) from {username='kudu', principal='kudu/127.27.180.190@KRBTEST.COM'} at 127.0.0.1:34488
I20260811 18:57:47.002619 23407 tablet_service.cc:1514] Processing DeleteTablet for tablet 8edc1d0ef4524b6087a0d283537ac75f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:57:46 UTC) from {username='kudu', principal='kudu/127.27.180.190@KRBTEST.COM'} at 127.0.0.1:47342
I20260811 18:57:47.002730 23290 tablet_replica.cc:330] T 8edc1d0ef4524b6087a0d283537ac75f P caf849fdfdc048128ec5b5502845312a: stopping tablet replica
I20260811 18:57:47.002903 23290 raft_consensus.cc:2229] T 8edc1d0ef4524b6087a0d283537ac75f P caf849fdfdc048128ec5b5502845312a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:47.003072 23290 raft_consensus.cc:2258] T 8edc1d0ef4524b6087a0d283537ac75f P caf849fdfdc048128ec5b5502845312a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:47.003460 23866 tablet_replica.cc:330] T 8edc1d0ef4524b6087a0d283537ac75f P 1efdcbeb44e94a3bb3acb45daa0e7197: stopping tablet replica
I20260811 18:57:47.003801 23866 raft_consensus.cc:2229] T 8edc1d0ef4524b6087a0d283537ac75f P 1efdcbeb44e94a3bb3acb45daa0e7197 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:57:47.004134 23866 raft_consensus.cc:2258] T 8edc1d0ef4524b6087a0d283537ac75f P 1efdcbeb44e94a3bb3acb45daa0e7197 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:47.004171 23290 ts_tablet_manager.cc:1905] T 8edc1d0ef4524b6087a0d283537ac75f P caf849fdfdc048128ec5b5502845312a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:47.004516 23870 tablet_replica.cc:330] T 8edc1d0ef4524b6087a0d283537ac75f P d37ed58d1e4e4273a7ad3a2a20681c90: stopping tablet replica
I20260811 18:57:47.004803 23870 raft_consensus.cc:2229] T 8edc1d0ef4524b6087a0d283537ac75f P d37ed58d1e4e4273a7ad3a2a20681c90 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:57:47.005162 23870 raft_consensus.cc:2258] T 8edc1d0ef4524b6087a0d283537ac75f P d37ed58d1e4e4273a7ad3a2a20681c90 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:57:47.006702 23870 ts_tablet_manager.cc:1905] T 8edc1d0ef4524b6087a0d283537ac75f P d37ed58d1e4e4273a7ad3a2a20681c90: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:47.006964 23290 ts_tablet_manager.cc:1918] T 8edc1d0ef4524b6087a0d283537ac75f P caf849fdfdc048128ec5b5502845312a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:57:47.007037 23290 log.cc:1196] T 8edc1d0ef4524b6087a0d283537ac75f P caf849fdfdc048128ec5b5502845312a: Deleting WAL directory at /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-0/wal/wals/8edc1d0ef4524b6087a0d283537ac75f
I20260811 18:57:47.007090 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 23015
I20260811 18:57:47.007289 23290 ts_tablet_manager.cc:1939] T 8edc1d0ef4524b6087a0d283537ac75f P caf849fdfdc048128ec5b5502845312a: Deleting consensus metadata
I20260811 18:57:47.031160 23870 ts_tablet_manager.cc:1918] T 8edc1d0ef4524b6087a0d283537ac75f P d37ed58d1e4e4273a7ad3a2a20681c90: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:57:47.031476 23870 log.cc:1196] T 8edc1d0ef4524b6087a0d283537ac75f P d37ed58d1e4e4273a7ad3a2a20681c90: Deleting WAL directory at /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-1/wal/wals/8edc1d0ef4524b6087a0d283537ac75f
I20260811 18:57:47.031939 23870 ts_tablet_manager.cc:1939] T 8edc1d0ef4524b6087a0d283537ac75f P d37ed58d1e4e4273a7ad3a2a20681c90: Deleting consensus metadata
I20260811 18:57:47.033190 23866 ts_tablet_manager.cc:1905] T 8edc1d0ef4524b6087a0d283537ac75f P 1efdcbeb44e94a3bb3acb45daa0e7197: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:57:47.036263 23866 ts_tablet_manager.cc:1918] T 8edc1d0ef4524b6087a0d283537ac75f P 1efdcbeb44e94a3bb3acb45daa0e7197: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:57:47.036382 23866 log.cc:1196] T 8edc1d0ef4524b6087a0d283537ac75f P 1efdcbeb44e94a3bb3acb45daa0e7197: Deleting WAL directory at /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-2/wal/wals/8edc1d0ef4524b6087a0d283537ac75f
I20260811 18:57:47.036697 23866 ts_tablet_manager.cc:1939] T 8edc1d0ef4524b6087a0d283537ac75f P 1efdcbeb44e94a3bb3acb45daa0e7197: Deleting consensus metadata
I20260811 18:57:47.037273 22915 catalog_manager.cc:4841] TS d37ed58d1e4e4273a7ad3a2a20681c90 (127.27.180.130:36347): tablet 8edc1d0ef4524b6087a0d283537ac75f (table test-table [id=a7f1d442a7124aeb8b69d6c9422884bc]) successfully deleted
I20260811 18:57:47.038102 22918 catalog_manager.cc:4841] TS 1efdcbeb44e94a3bb3acb45daa0e7197 (127.27.180.131:45203): tablet 8edc1d0ef4524b6087a0d283537ac75f (table test-table [id=a7f1d442a7124aeb8b69d6c9422884bc]) successfully deleted
W20260811 18:57:47.046468 22914 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.27.180.129:41443 (error 108)
I20260811 18:57:47.047065 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 23282
I20260811 18:57:47.058775 23288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:47.056753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.129:41443 (local address 127.0.0.1:34274)
0811 18:57:47.058567 (+ 1814us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:47.058662 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.129:41443: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1714}
W20260811 18:57:47.060757 22914 catalog_manager.cc:4584] TS caf849fdfdc048128ec5b5502845312a (127.27.180.129:41443): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8edc1d0ef4524b6087a0d283537ac75f: Network error: Client connection negotiation failed: client connection to 127.27.180.129:41443: connect: Connection refused (error 111)
I20260811 18:57:47.066785 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 23541
I20260811 18:57:47.082383 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 22870
I20260811 18:57:47.093164 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.190:40989
--webserver_interface=127.27.180.190
--webserver_port=37621
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab
--principal=kudu/127.27.180.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:42327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.180.190:40989
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:57:47.205302 23872 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:57:47.205590 23872 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:57:47.205649 23872 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:57:47.209710 23872 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:57:47.209771 23872 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:57:47.209798 23872 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:57:47.209820 23872 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:57:47.209846 23872 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:57:47.215262 23872 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:42327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.180.190:40989
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab
--principal=kudu/127.27.180.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.190:40989
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.180.190
--webserver_port=37621
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23872
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:57:47.216368 23872 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:57:47.217446 23872 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:57:47.226202 23883 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:47.226207 23886 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:47.226207 23882 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:57:47.226624 23872 server_base.cc:1034] running on GCE node
I20260811 18:57:47.227203 23872 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:57:47.228389 23872 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:57:47.229579 23872 hybrid_clock.cc:648] HybridClock initialized: now 1786474667229549 us; error 43 us; skew 500 ppm
Aug 11 18:57:47 dist-test-slave-2j7r krb5kdc[21860](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474667, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:57:47.232504 23872 init.cc:377] Logged in from keytab as kudu/127.27.180.190@KRBTEST.COM (short username kudu)
I20260811 18:57:47.233500 23872 webserver.cc:460] Webserver started at http://127.27.180.190:37621/ using document root <none> and password file <none>
I20260811 18:57:47.234062 23872 fs_manager.cc:359] Metadata directory not provided
I20260811 18:57:47.234134 23872 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:57:47.238627 23872 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 18:57:47.241159 23897 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:47.242652 23872 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:57:47.242805 23872 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/master-0/wal
uuid: "46e6feef897141feb6bf6aba7a68a7ce"
format_stamp: "Formatted at 2026-08-11 18:57:45 on dist-test-slave-2j7r"
server_key: "446f7ff2aba1006e44a2e51af698b0e1"
server_key_iv: "409541b09efda84ff9e45bf5477e5af7"
server_key_version: "encryptionkey@0"
I20260811 18:57:47.243395 23872 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:57:47.252583 23872 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:57:47.255622 23872 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:57:47.255800 23872 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:57:47.263084 23872 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.190:40989
I20260811 18:57:47.263127 24011 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.190:40989 every 8 connection(s)
I20260811 18:57:47.264153 23872 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
I20260811 18:57:47.268564 24014 sys_catalog.cc:263] Verifying existing consensus state
I20260811 18:57:47.269970 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 23872
I20260811 18:57:47.270404 24014 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46e6feef897141feb6bf6aba7a68a7ce: Bootstrap starting.
I20260811 18:57:47.270424 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.180.129:41443
--local_ip_for_outbound_sockets=127.27.180.129
--tserver_master_addrs=127.27.180.190:40989
--webserver_port=46263
--webserver_interface=127.27.180.129
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.129.keytab
--principal=kudu/127.27.180.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:42327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 18:57:47.294103 24014 log.cc:824] T 00000000000000000000000000000000 P 46e6feef897141feb6bf6aba7a68a7ce: Log is configured to *not* fsync() on all Append() calls
I20260811 18:57:47.305807 24014 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46e6feef897141feb6bf6aba7a68a7ce: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 18:57:47.306200 24014 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46e6feef897141feb6bf6aba7a68a7ce: Bootstrap complete.
I20260811 18:57:47.310400 24014 raft_consensus.cc:348] T 00000000000000000000000000000000 P 46e6feef897141feb6bf6aba7a68a7ce [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46e6feef897141feb6bf6aba7a68a7ce" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 40989 } }
I20260811 18:57:47.310794 24014 raft_consensus.cc:729] T 00000000000000000000000000000000 P 46e6feef897141feb6bf6aba7a68a7ce [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46e6feef897141feb6bf6aba7a68a7ce, State: Initialized, Role: FOLLOWER
I20260811 18:57:47.311277 24014 consensus_queue.cc:260] T 00000000000000000000000000000000 P 46e6feef897141feb6bf6aba7a68a7ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46e6feef897141feb6bf6aba7a68a7ce" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 40989 } }
I20260811 18:57:47.311399 24014 raft_consensus.cc:388] T 00000000000000000000000000000000 P 46e6feef897141feb6bf6aba7a68a7ce [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:57:47.311463 24014 raft_consensus.cc:482] T 00000000000000000000000000000000 P 46e6feef897141feb6bf6aba7a68a7ce [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:57:47.311553 24014 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 46e6feef897141feb6bf6aba7a68a7ce [term 2 FOLLOWER]: Advancing to term 3
I20260811 18:57:47.313455 24014 raft_consensus.cc:504] T 00000000000000000000000000000000 P 46e6feef897141feb6bf6aba7a68a7ce [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46e6feef897141feb6bf6aba7a68a7ce" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 40989 } }
I20260811 18:57:47.313768 24014 leader_election.cc:302] T 00000000000000000000000000000000 P 46e6feef897141feb6bf6aba7a68a7ce [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 46e6feef897141feb6bf6aba7a68a7ce; no voters:
I20260811 18:57:47.314069 24014 leader_election.cc:288] T 00000000000000000000000000000000 P 46e6feef897141feb6bf6aba7a68a7ce [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 18:57:47.314203 24020 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 46e6feef897141feb6bf6aba7a68a7ce [term 3 FOLLOWER]: Leader election won for term 3
I20260811 18:57:47.314447 24020 raft_consensus.cc:686] T 00000000000000000000000000000000 P 46e6feef897141feb6bf6aba7a68a7ce [term 3 LEADER]: Becoming Leader. State: Replica: 46e6feef897141feb6bf6aba7a68a7ce, State: Running, Role: LEADER
I20260811 18:57:47.314752 24020 consensus_queue.cc:237] T 00000000000000000000000000000000 P 46e6feef897141feb6bf6aba7a68a7ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46e6feef897141feb6bf6aba7a68a7ce" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 40989 } }
I20260811 18:57:47.315227 24014 sys_catalog.cc:564] T 00000000000000000000000000000000 P 46e6feef897141feb6bf6aba7a68a7ce [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:57:47.316275 24023 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46e6feef897141feb6bf6aba7a68a7ce [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "46e6feef897141feb6bf6aba7a68a7ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46e6feef897141feb6bf6aba7a68a7ce" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 40989 } } }
I20260811 18:57:47.316419 24023 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46e6feef897141feb6bf6aba7a68a7ce [sys.catalog]: This master's current role is: LEADER
I20260811 18:57:47.316288 24026 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46e6feef897141feb6bf6aba7a68a7ce [sys.catalog]: SysCatalogTable state changed. Reason: New leader 46e6feef897141feb6bf6aba7a68a7ce. Latest consensus state: current_term: 3 leader_uuid: "46e6feef897141feb6bf6aba7a68a7ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46e6feef897141feb6bf6aba7a68a7ce" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 40989 } } }
I20260811 18:57:47.316704 24026 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46e6feef897141feb6bf6aba7a68a7ce [sys.catalog]: This master's current role is: LEADER
I20260811 18:57:47.320202 24053 catalog_manager.cc:1259] Loaded cluster ID: fe5b4857e0284e22ba246b0656cb5426
I20260811 18:57:47.320261 24053 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 46e6feef897141feb6bf6aba7a68a7ce: loading cluster ID for follower catalog manager: success
W20260811 18:57:47.321375 24053 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 46e6feef897141feb6bf6aba7a68a7ce: acquiring CA information for follower catalog manager: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
W20260811 18:57:47.321448 24053 catalog_manager.cc:873] Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO: failed to prepare follower catalog manager, will retry
I20260811 18:57:47.321508 24039 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:57:47.323575 24039 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:57:47.324084 24039 catalog_manager.cc:1259] Loaded cluster ID: fe5b4857e0284e22ba246b0656cb5426
I20260811 18:57:47.324146 24039 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260811 18:57:47.324828 24039 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f2a4e4c19ad google::LogMessage::Fail() at ??:0
@ 0x7f2a4e4c488a google::LogMessage::SendToLog() at ??:0
@ 0x7f2a4e4c14de google::LogMessage::Flush() at ??:0
@ 0x7f2a4e4c2a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f2a58e00fdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f2a58e01d2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f2a58dfe579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f2a58e42640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f2a507133d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f2a50713c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f2a507153f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f2a50704512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f2a51d486db start_thread at ??:0
@ 0x7f2a4cfbf61f clone at ??:0
W20260811 18:57:47.404246 24017 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:57:47.404492 24017 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:57:47.404534 24017 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:57:47.408771 24017 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:57:47.408849 24017 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:57:47.408928 24017 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.180.129
I20260811 18:57:47.415581 24017 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:42327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.129.keytab
--principal=kudu/127.27.180.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.180.129:41443
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.180.129
--webserver_port=46263
--tserver_master_addrs=127.27.180.190:40989
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24017
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.180.129
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:57:47.416801 24017 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:57:47.417842 24017 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:57:47.427263 24095 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:47.427245 24091 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:57:47.427536 24017 server_base.cc:1034] running on GCE node
W20260811 18:57:47.427245 24088 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:57:47.428009 24017 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:57:47.428637 24017 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:57:47.429804 24017 hybrid_clock.cc:648] HybridClock initialized: now 1786474667429776 us; error 45 us; skew 500 ppm
Aug 11 18:57:47 dist-test-slave-2j7r krb5kdc[21860](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474667, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:57:47.432391 24017 init.cc:377] Logged in from keytab as kudu/127.27.180.129@KRBTEST.COM (short username kudu)
I20260811 18:57:47.433228 24017 webserver.cc:460] Webserver started at http://127.27.180.129:46263/ using document root <none> and password file <none>
I20260811 18:57:47.433753 24017 fs_manager.cc:359] Metadata directory not provided
I20260811 18:57:47.433813 24017 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:57:47.437664 24017 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:57:47.439817 24111 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:47.441108 24017 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 18:57:47.441226 24017 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-0/wal
uuid: "caf849fdfdc048128ec5b5502845312a"
format_stamp: "Formatted at 2026-08-11 18:57:45 on dist-test-slave-2j7r"
server_key: "9bf6192df9a7f04098127322b8a2ef92"
server_key_iv: "bc7f2259ae066343b8817f074423818a"
server_key_version: "encryptionkey@0"
I20260811 18:57:47.441778 24017 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:57:47.467917 24017 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:57:47.471192 24017 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:57:47.471401 24017 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:57:47.472128 24017 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:57:47.473029 24017 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:57:47.473124 24017 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:47.473191 24017 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:57:47.473227 24017 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:47.483215 24017 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.129:41443
I20260811 18:57:47.483238 24324 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.129:41443 every 8 connection(s)
I20260811 18:57:47.484432 24017 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-0/data/info.pb
I20260811 18:57:47.489249 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 24017
I20260811 18:57:47.489635 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.180.130:36347
--local_ip_for_outbound_sockets=127.27.180.130
--tserver_master_addrs=127.27.180.190:40989
--webserver_port=40669
--webserver_interface=127.27.180.130
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.130.keytab
--principal=kudu/127.27.180.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:42327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:57:47.599414 24331 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:57:47.599656 24331 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:57:47.599725 24331 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:57:47.603628 24331 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:57:47.603679 24331 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:57:47.603762 24331 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.180.130
I20260811 18:57:47.608897 24331 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:42327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.130.keytab
--principal=kudu/127.27.180.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.180.130:36347
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.180.130
--webserver_port=40669
--tserver_master_addrs=127.27.180.190:40989
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24331
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.180.130
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:57:47.609910 24331 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:57:47.610894 24331 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:57:47.620311 24345 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:47.620316 24340 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:47.620312 24342 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:57:47.620520 24331 server_base.cc:1034] running on GCE node
I20260811 18:57:47.621021 24331 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:57:47.621654 24331 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:57:47.622828 24331 hybrid_clock.cc:648] HybridClock initialized: now 1786474667622812 us; error 31 us; skew 500 ppm
Aug 11 18:57:47 dist-test-slave-2j7r krb5kdc[21860](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474667, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:57:47.625550 24331 init.cc:377] Logged in from keytab as kudu/127.27.180.130@KRBTEST.COM (short username kudu)
I20260811 18:57:47.626431 24331 webserver.cc:460] Webserver started at http://127.27.180.130:40669/ using document root <none> and password file <none>
I20260811 18:57:47.626958 24331 fs_manager.cc:359] Metadata directory not provided
I20260811 18:57:47.627025 24331 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:57:47.630863 24331 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:57:47.632952 24360 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:47.634213 24331 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 18:57:47.634321 24331 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-1/wal
uuid: "d37ed58d1e4e4273a7ad3a2a20681c90"
format_stamp: "Formatted at 2026-08-11 18:57:45 on dist-test-slave-2j7r"
server_key: "15916bcb143779610fea979bb97c7f57"
server_key_iv: "e234587a7b46cce4a2a22c70a0ec694d"
server_key_version: "encryptionkey@0"
I20260811 18:57:47.634846 24331 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
W20260811 18:57:47.652007 24330 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:57:47.486751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:40989 (local address 127.27.180.129:51157)
0811 18:57:47.487258 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:47.487300 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 18:57:47.488503 (+ 1203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:57:47.651058 (+162555us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:40989: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":149,"threads_started":1}
W20260811 18:57:47.652559 24325 heartbeater.cc:644] Failed to heartbeat to 127.27.180.190:40989 (0 consecutive failures): Network error: Failed to ping master at 127.27.180.190:40989: Client connection negotiation failed: client connection to 127.27.180.190:40989: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 18:57:47.653355 24330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:47.653025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:40989 (local address 127.27.180.129:47593)
0811 18:57:47.653204 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:47.653290 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260811 18:57:47.653954 24330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:47.653767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:40989 (local address 127.27.180.129:37091)
0811 18:57:47.653854 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:47.653899 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260811 18:57:47.654090 24325 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 18:57:47.667697 24331 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:57:47.670957 24331 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:57:47.671136 24331 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:57:47.671746 24331 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:57:47.672655 24331 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:57:47.672709 24331 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:47.672753 24331 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:57:47.672777 24331 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:47.682039 24331 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.130:36347
I20260811 18:57:47.682063 24566 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.130:36347 every 8 connection(s)
I20260811 18:57:47.683024 24331 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-1/data/info.pb
I20260811 18:57:47.686964 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 24331
I20260811 18:57:47.686998 24574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:47.685156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:40989 (local address 127.27.180.130:51363)
0811 18:57:47.685603 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:47.685720 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":107,"threads_started":1}
I20260811 18:57:47.687409 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.180.131:45203
--local_ip_for_outbound_sockets=127.27.180.131
--tserver_master_addrs=127.27.180.190:40989
--webserver_port=46285
--webserver_interface=127.27.180.131
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.131.keytab
--principal=kudu/127.27.180.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:42327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:57:47.687460 24567 heartbeater.cc:644] Failed to heartbeat to 127.27.180.190:40989 (0 consecutive failures): Network error: Failed to ping master at 127.27.180.190:40989: Client connection negotiation failed: client connection to 127.27.180.190:40989: connect: Connection refused (error 111)
I20260811 18:57:47.688047 24574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:47.687838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:40989 (local address 127.27.180.130:39603)
0811 18:57:47.687947 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:47.687996 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260811 18:57:47.688619 24574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:47.688457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:40989 (local address 127.27.180.130:60579)
0811 18:57:47.688531 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:47.688570 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260811 18:57:47.688788 24567 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 18:57:47.798125 24578 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:57:47.798359 24578 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:57:47.798400 24578 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:57:47.802342 24578 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:57:47.802392 24578 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:57:47.802461 24578 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.180.131
I20260811 18:57:47.807513 24578 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:42327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.131.keytab
--principal=kudu/127.27.180.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.180.131:45203
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.180.131
--webserver_port=46285
--tserver_master_addrs=127.27.180.190:40989
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24578
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.180.131
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:57:47.808478 24578 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:57:47.809454 24578 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:57:47.818938 24595 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:47.818957 24584 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:57:47.818969 24590 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:57:47.819190 24578 server_base.cc:1034] running on GCE node
I20260811 18:57:47.819581 24578 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:57:47.820181 24578 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:57:47.821343 24578 hybrid_clock.cc:648] HybridClock initialized: now 1786474667821329 us; error 24 us; skew 500 ppm
Aug 11 18:57:47 dist-test-slave-2j7r krb5kdc[21860](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474667, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:57:47.824086 24578 init.cc:377] Logged in from keytab as kudu/127.27.180.131@KRBTEST.COM (short username kudu)
I20260811 18:57:47.824931 24578 webserver.cc:460] Webserver started at http://127.27.180.131:46285/ using document root <none> and password file <none>
I20260811 18:57:47.825498 24578 fs_manager.cc:359] Metadata directory not provided
I20260811 18:57:47.825567 24578 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:57:47.829291 24578 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260811 18:57:47.831251 24610 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:47.832381 24578 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 18:57:47.832508 24578 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-2/wal
uuid: "1efdcbeb44e94a3bb3acb45daa0e7197"
format_stamp: "Formatted at 2026-08-11 18:57:45 on dist-test-slave-2j7r"
server_key: "702b7bf9382ecdd0b10b3c3fd01da645"
server_key_iv: "5326459e8b34be2c10b5b59c40bac440"
server_key_version: "encryptionkey@0"
I20260811 18:57:47.833017 24578 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:57:47.852203 24578 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:57:47.855489 24578 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:57:47.855695 24578 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:57:47.856320 24578 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:57:47.857226 24578 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:57:47.857282 24578 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:47.857334 24578 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:57:47.857368 24578 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:57:47.866645 24578 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.131:45203
I20260811 18:57:47.866667 24813 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.131:45203 every 8 connection(s)
I20260811 18:57:47.867684 24578 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786474496157106-28370-0/minicluster-data/ts-2/data/info.pb
I20260811 18:57:47.871627 24816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:47.869837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:40989 (local address 127.27.180.131:37463)
0811 18:57:47.870350 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:47.870475 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":140,"threads_started":1}
W20260811 18:57:47.872094 24814 heartbeater.cc:644] Failed to heartbeat to 127.27.180.190:40989 (0 consecutive failures): Network error: Failed to ping master at 127.27.180.190:40989: Client connection negotiation failed: client connection to 127.27.180.190:40989: connect: Connection refused (error 111)
I20260811 18:57:47.872695 24816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:47.872472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:40989 (local address 127.27.180.131:41865)
0811 18:57:47.872581 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:47.872638 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260811 18:57:47.873306 24816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:47.873123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:40989 (local address 127.27.180.131:56563)
0811 18:57:47.873208 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:47.873255 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260811 18:57:47.873456 24814 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 18:57:47.874572 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 24578
I20260811 18:57:48.655017 24819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:48.654603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:40989 (local address 127.27.180.129:52817)
0811 18:57:48.654845 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:48.654899 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":73,"threads_started":1}
I20260811 18:57:48.689713 24822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:48.689328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:40989 (local address 127.27.180.130:51665)
0811 18:57:48.689558 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:48.689611 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":71,"threads_started":1}
I20260811 18:57:48.874498 24823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:48.873998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:40989 (local address 127.27.180.131:56207)
0811 18:57:48.874312 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:48.874373 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":78,"threads_started":1}
I20260811 18:57:49.656337 24824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:49.655845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:40989 (local address 127.27.180.129:57167)
0811 18:57:49.656137 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:49.656208 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":99,"threads_started":1}
I20260811 18:57:49.690752 24825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:49.690375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:40989 (local address 127.27.180.130:38547)
0811 18:57:49.690598 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:49.690652 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":72,"threads_started":1}
I20260811 18:57:49.875732 24826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:49.875285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:40989 (local address 127.27.180.131:39233)
0811 18:57:49.875568 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:49.875633 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":84,"threads_started":1}
I20260811 18:57:50.657591 24827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:50.657161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:40989 (local address 127.27.180.129:43493)
0811 18:57:50.657420 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:50.657484 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":83,"threads_started":1}
I20260811 18:57:50.691888 24829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:50.691437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:40989 (local address 127.27.180.130:46863)
0811 18:57:50.691722 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:50.691786 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":76,"threads_started":1}
I20260811 18:57:50.876873 24832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:50.876457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:40989 (local address 127.27.180.131:59989)
0811 18:57:50.876717 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:50.876779 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":78,"threads_started":1}
I20260811 18:57:51.658707 24837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:51.658292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:40989 (local address 127.27.180.129:57085)
0811 18:57:51.658542 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:51.658597 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":78,"threads_started":1}
I20260811 18:57:51.693020 24838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:51.692594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:40989 (local address 127.27.180.130:56967)
0811 18:57:51.692873 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:51.692927 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":97,"threads_started":1}
I20260811 18:57:51.878033 24839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:51.877586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:40989 (local address 127.27.180.131:55437)
0811 18:57:51.877868 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:51.877926 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":79,"threads_started":1}
I20260811 18:57:52.659912 24842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:52.659481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:40989 (local address 127.27.180.129:37507)
0811 18:57:52.659744 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:52.659806 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":81,"threads_started":1}
I20260811 18:57:52.694155 24843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:52.693759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:40989 (local address 127.27.180.130:39279)
0811 18:57:52.694004 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:52.694061 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":74,"threads_started":1}
I20260811 18:57:52.879179 24845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:52.878782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:40989 (local address 127.27.180.131:52945)
0811 18:57:52.879035 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:52.879088 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":88,"threads_started":1}
I20260811 18:57:53.663434 24846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:53.660567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:40989 (local address 127.27.180.129:35869)
0811 18:57:53.663201 (+ 2634us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:53.663296 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2534,"thread_start_us":2430,"threads_started":1}
I20260811 18:57:53.695398 24849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:53.694895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:40989 (local address 127.27.180.130:55165)
0811 18:57:53.695174 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:53.695280 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":88,"threads_started":1}
I20260811 18:57:53.880419 24850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:53.879927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:40989 (local address 127.27.180.131:34441)
0811 18:57:53.880240 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:53.880311 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":79,"threads_started":1}
I20260811 18:57:54.666393 24851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:54.665883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:40989 (local address 127.27.180.129:56709)
0811 18:57:54.666227 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:54.666288 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":72,"threads_started":1}
I20260811 18:57:54.696589 24852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:54.696111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:40989 (local address 127.27.180.130:54181)
0811 18:57:54.696425 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:54.696483 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":73,"threads_started":1}
I20260811 18:57:54.881721 24854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:54.881213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:40989 (local address 127.27.180.131:39751)
0811 18:57:54.881543 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:54.881603 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":74,"threads_started":1}
I20260811 18:57:55.667621 24855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:55.667178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:40989 (local address 127.27.180.129:43071)
0811 18:57:55.667455 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:55.667519 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":90,"threads_started":1}
I20260811 18:57:55.697739 24856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:55.697326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:40989 (local address 127.27.180.130:42859)
0811 18:57:55.697584 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:55.697642 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":83,"threads_started":1}
I20260811 18:57:55.882894 24862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:55.882452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:40989 (local address 127.27.180.131:50585)
0811 18:57:55.882723 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:55.882796 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":73,"threads_started":1}
I20260811 18:57:56.668821 24864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:56.668382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:40989 (local address 127.27.180.129:45165)
0811 18:57:56.668639 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:56.668700 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":79,"threads_started":1}
I20260811 18:57:56.698835 24865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:56.698417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:40989 (local address 127.27.180.130:34185)
0811 18:57:56.698675 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:56.698730 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":76,"threads_started":1}
I20260811 18:57:56.884055 24866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:56.883628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:40989 (local address 127.27.180.131:49977)
0811 18:57:56.883898 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:56.883958 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":78,"threads_started":1}
I20260811 18:57:57.670063 24868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:57.669633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:40989 (local address 127.27.180.129:41243)
0811 18:57:57.669901 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:57.669961 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":76,"threads_started":1}
I20260811 18:57:57.699983 24871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:57.699549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:40989 (local address 127.27.180.130:39453)
0811 18:57:57.699822 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:57.699881 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":88,"threads_started":1}
I20260811 18:57:57.885370 24872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:57.884853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:40989 (local address 127.27.180.131:49605)
0811 18:57:57.885190 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:57.885259 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":83,"threads_started":1}
I20260811 18:57:58.671247 24873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:58.670839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:40989 (local address 127.27.180.129:47407)
0811 18:57:58.671082 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:58.671138 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":76,"threads_started":1}
I20260811 18:57:58.701078 24874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:58.700664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:40989 (local address 127.27.180.130:57389)
0811 18:57:58.700933 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:58.700985 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":78,"threads_started":1}
I20260811 18:57:58.886552 24875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:58.886089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:40989 (local address 127.27.180.131:45305)
0811 18:57:58.886397 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:58.886453 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":126,"threads_started":1}
I20260811 18:57:59.672355 24876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:59.671936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:40989 (local address 127.27.180.129:41327)
0811 18:57:59.672176 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:59.672253 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":78,"threads_started":1}
I20260811 18:57:59.702206 24878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:59.701814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:40989 (local address 127.27.180.130:47167)
0811 18:57:59.702052 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:59.702108 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":74,"threads_started":1}
I20260811 18:57:59.889286 24879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:57:59.887466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:40989 (local address 127.27.180.131:57333)
0811 18:57:59.889113 (+ 1647us) negotiation.cc:107] Waiting for socket to connect
0811 18:57:59.889178 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1548,"thread_start_us":1451,"threads_started":1}
I20260811 18:58:00.673547 24880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:00.673125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:40989 (local address 127.27.180.129:42939)
0811 18:58:00.673387 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:00.673445 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":74,"threads_started":1}
I20260811 18:58:00.703317 24881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:00.702925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:40989 (local address 127.27.180.130:36395)
0811 18:58:00.703158 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:00.703214 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":76,"threads_started":1}
I20260811 18:58:00.890555 24884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:00.890134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:40989 (local address 127.27.180.131:51867)
0811 18:58:00.890398 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:00.890457 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":75,"threads_started":1}
I20260811 18:58:01.674649 24885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:01.674193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:40989 (local address 127.27.180.129:34821)
0811 18:58:01.674495 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:01.674551 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":110,"threads_started":1}
I20260811 18:58:01.704564 24886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:01.704084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:40989 (local address 127.27.180.130:38033)
0811 18:58:01.704391 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:01.704457 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":80,"threads_started":1}
I20260811 18:58:01.891731 24888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:01.891214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:40989 (local address 127.27.180.131:33705)
0811 18:58:01.891547 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:01.891612 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":83,"threads_started":1}
I20260811 18:58:02.676023 24890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:02.675543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:40989 (local address 127.27.180.129:60563)
0811 18:58:02.675833 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:02.675900 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":76,"threads_started":1}
I20260811 18:58:02.705960 24898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:02.705489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:40989 (local address 127.27.180.130:39797)
0811 18:58:02.705774 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:02.705838 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":74,"threads_started":1}
I20260811 18:58:02.882865 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 24017
I20260811 18:58:02.895172 24899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:02.894746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:40989 (local address 127.27.180.131:40641)
0811 18:58:02.895009 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:02.895071 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:40989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":82,"threads_started":1}
I20260811 18:58:02.900009 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 24331
I20260811 18:58:02.912186 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 24578
2026-08-11T18:58:02Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19339 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:58:02 dist-test-slave-2j7r krb5kdc[24908](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:58:02 dist-test-slave-2j7r krb5kdc[24908](info): set up 2 sockets
Aug 11 18:58:02 dist-test-slave-2j7r krb5kdc[24908](info): commencing operation
krb5kdc: starting...
W20260811 18:58:04.118984 28370 mini_kdc.cc:121] Time spent starting KDC: real 1.178s user 0.007s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/krb5kdc/test-user.keytab.
Aug 11 18:58:04 dist-test-slave-2j7r krb5kdc[24908](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474684, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:58:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:58:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.180.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.180.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.180.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab.
Entry for principal kudu/127.27.180.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab.
I20260811 18:58:04.244742 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.190:33249
--webserver_interface=127.27.180.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab
--principal=kudu/127.27.180.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:34601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.180.190:33249
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:58:04.359267 24931 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:58:04.359558 24931 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:58:04.359601 24931 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:58:04.363749 24931 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 18:58:04.363816 24931 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 18:58:04.363830 24931 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 18:58:04.363843 24931 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:58:04.363860 24931 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:58:04.363876 24931 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:58:04.363894 24931 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:58:04.363919 24931 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:58:04.369354 24931 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:34601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.27.180.190:33249
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab
--principal=kudu/127.27.180.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.190:33249
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.180.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24931
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:58:04.370452 24931 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:58:04.371532 24931 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:58:04.380481 24938 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:04.380522 24941 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:04.380499 24943 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:58:04.380652 24931 server_base.cc:1034] running on GCE node
I20260811 18:58:04.381434 24931 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:58:04.382586 24931 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:58:04.383775 24931 hybrid_clock.cc:648] HybridClock initialized: now 1786474684383747 us; error 38 us; skew 500 ppm
Aug 11 18:58:04 dist-test-slave-2j7r krb5kdc[24908](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474684, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:58:04.386766 24931 init.cc:377] Logged in from keytab as kudu/127.27.180.190@KRBTEST.COM (short username kudu)
I20260811 18:58:04.387681 24931 webserver.cc:460] Webserver started at http://127.27.180.190:33175/ using document root <none> and password file <none>
I20260811 18:58:04.388219 24931 fs_manager.cc:359] Metadata directory not provided
I20260811 18:58:04.388281 24931 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:58:04.388499 24931 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:58:04.390636 24931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/master-0/data/instance:
uuid: "b3054cf295fb43d6bb0787856013df0e"
format_stamp: "Formatted at 2026-08-11 18:58:04 on dist-test-slave-2j7r"
server_key: "7cb1e0c8cf224b542b7cef8b2d22b75d"
server_key_iv: "4b3cfb7012c4f74187c871d278b3bb3b"
server_key_version: "encryptionkey@0"
I20260811 18:58:04.391197 24931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/master-0/wal/instance:
uuid: "b3054cf295fb43d6bb0787856013df0e"
format_stamp: "Formatted at 2026-08-11 18:58:04 on dist-test-slave-2j7r"
server_key: "7cb1e0c8cf224b542b7cef8b2d22b75d"
server_key_iv: "4b3cfb7012c4f74187c871d278b3bb3b"
server_key_version: "encryptionkey@0"
I20260811 18:58:04.394858 24931 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 18:58:04.397221 24953 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:04.398453 24931 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 18:58:04.398569 24931 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/master-0/wal
uuid: "b3054cf295fb43d6bb0787856013df0e"
format_stamp: "Formatted at 2026-08-11 18:58:04 on dist-test-slave-2j7r"
server_key: "7cb1e0c8cf224b542b7cef8b2d22b75d"
server_key_iv: "4b3cfb7012c4f74187c871d278b3bb3b"
server_key_version: "encryptionkey@0"
I20260811 18:58:04.398663 24931 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:58:04.413763 24931 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:58:04.416469 24931 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:58:04.416647 24931 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:58:04.426082 24931 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.190:33249
I20260811 18:58:04.426388 25043 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.190:33249 every 8 connection(s)
I20260811 18:58:04.427038 24931 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
I20260811 18:58:04.429785 25046 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:58:04.432034 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 24931
I20260811 18:58:04.432171 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/master-0/wal/instance
I20260811 18:58:04.432500 28370 external_mini_cluster.cc:1428] Setting key 569bcae2e508617e0156c5a107089d77
I20260811 18:58:04.436180 25046 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b3054cf295fb43d6bb0787856013df0e: Bootstrap starting.
I20260811 18:58:04.438841 25046 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b3054cf295fb43d6bb0787856013df0e: Neither blocks nor log segments found. Creating new log.
I20260811 18:58:04.439669 25046 log.cc:824] T 00000000000000000000000000000000 P b3054cf295fb43d6bb0787856013df0e: Log is configured to *not* fsync() on all Append() calls
Aug 11 18:58:04 dist-test-slave-2j7r krb5kdc[24908](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474684, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM
I20260811 18:58:04.441982 25046 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b3054cf295fb43d6bb0787856013df0e: No bootstrap required, opened a new log
I20260811 18:58:04.446107 25046 raft_consensus.cc:348] T 00000000000000000000000000000000 P b3054cf295fb43d6bb0787856013df0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3054cf295fb43d6bb0787856013df0e" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 33249 } }
I20260811 18:58:04.446349 25046 raft_consensus.cc:374] T 00000000000000000000000000000000 P b3054cf295fb43d6bb0787856013df0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:58:04.446408 25046 raft_consensus.cc:729] T 00000000000000000000000000000000 P b3054cf295fb43d6bb0787856013df0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3054cf295fb43d6bb0787856013df0e, State: Initialized, Role: FOLLOWER
I20260811 18:58:04.446794 25052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:04.433916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54008 (local address 127.27.180.190:33249)
0811 18:58:04.434344 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:04.434361 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:04.434397 (+ 36us) server_negotiation.cc:407] Connection header received
0811 18:58:04.435185 (+ 788us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:04.435202 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:04.435571 (+ 369us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:04.435851 (+ 280us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:04.436742 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:04.437779 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:04.438920 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:04.439404 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:04.439522 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:04.442206 (+ 2684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:58:04.442212 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:58:04.442225 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:58:04.442252 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:58:04.443872 (+ 1620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:04.444380 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:04.444385 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:04.444389 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:04.444499 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:04.444760 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:04.444762 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:04.444763 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:04.445185 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:58:04.445377 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:04.445620 (+ 243us) server_negotiation.cc:299] Negotiation successful
0811 18:58:04.445893 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":123,"threads_started":1}
I20260811 18:58:04.446827 25046 consensus_queue.cc:260] T 00000000000000000000000000000000 P b3054cf295fb43d6bb0787856013df0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3054cf295fb43d6bb0787856013df0e" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 33249 } }
I20260811 18:58:04.446981 25046 raft_consensus.cc:388] T 00000000000000000000000000000000 P b3054cf295fb43d6bb0787856013df0e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:58:04.447041 25046 raft_consensus.cc:482] T 00000000000000000000000000000000 P b3054cf295fb43d6bb0787856013df0e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:58:04.447140 25046 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b3054cf295fb43d6bb0787856013df0e [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:58:04.448035 25046 raft_consensus.cc:504] T 00000000000000000000000000000000 P b3054cf295fb43d6bb0787856013df0e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3054cf295fb43d6bb0787856013df0e" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 33249 } }
I20260811 18:58:04.448360 25046 leader_election.cc:302] T 00000000000000000000000000000000 P b3054cf295fb43d6bb0787856013df0e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b3054cf295fb43d6bb0787856013df0e; no voters:
I20260811 18:58:04.448657 25046 leader_election.cc:288] T 00000000000000000000000000000000 P b3054cf295fb43d6bb0787856013df0e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:58:04.448772 25057 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b3054cf295fb43d6bb0787856013df0e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:58:04.449036 25057 raft_consensus.cc:686] T 00000000000000000000000000000000 P b3054cf295fb43d6bb0787856013df0e [term 1 LEADER]: Becoming Leader. State: Replica: b3054cf295fb43d6bb0787856013df0e, State: Running, Role: LEADER
I20260811 18:58:04.449411 25057 consensus_queue.cc:237] T 00000000000000000000000000000000 P b3054cf295fb43d6bb0787856013df0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3054cf295fb43d6bb0787856013df0e" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 33249 } }
I20260811 18:58:04.449883 25046 sys_catalog.cc:564] T 00000000000000000000000000000000 P b3054cf295fb43d6bb0787856013df0e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:58:04.450963 25059 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3054cf295fb43d6bb0787856013df0e [sys.catalog]: SysCatalogTable state changed. Reason: New leader b3054cf295fb43d6bb0787856013df0e. Latest consensus state: current_term: 1 leader_uuid: "b3054cf295fb43d6bb0787856013df0e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3054cf295fb43d6bb0787856013df0e" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 33249 } } }
I20260811 18:58:04.451076 25059 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3054cf295fb43d6bb0787856013df0e [sys.catalog]: This master's current role is: LEADER
I20260811 18:58:04.451486 25058 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3054cf295fb43d6bb0787856013df0e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b3054cf295fb43d6bb0787856013df0e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3054cf295fb43d6bb0787856013df0e" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 33249 } } }
I20260811 18:58:04.451642 25058 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3054cf295fb43d6bb0787856013df0e [sys.catalog]: This master's current role is: LEADER
I20260811 18:58:04.451555 25071 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:58:04.454980 25071 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:58:04.460642 25071 catalog_manager.cc:1347] Generated new cluster ID: a07c061746b84cfda0f8ef5a97059df7
I20260811 18:58:04.460703 25071 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:58:04.477957 25071 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:58:04.479358 25071 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:58:04.484705 25071 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b3054cf295fb43d6bb0787856013df0e: Generated new TSK 0
I20260811 18:58:04.485487 25071 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.180.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.180.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.180.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.129.keytab.
Entry for principal kudu/127.27.180.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.129.keytab.
I20260811 18:58:04.522541 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.180.129:0
--local_ip_for_outbound_sockets=127.27.180.129
--webserver_interface=127.27.180.129
--webserver_port=0
--tserver_master_addrs=127.27.180.190:33249
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.129.keytab
--principal=kudu/127.27.180.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:34601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:58:04.644522 25090 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:58:04.644790 25090 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:58:04.644842 25090 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:58:04.648842 25090 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:58:04.648908 25090 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:58:04.648967 25090 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.180.129
I20260811 18:58:04.654345 25090 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:34601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.129.keytab
--principal=kudu/127.27.180.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.180.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.180.129
--webserver_port=0
--tserver_master_addrs=127.27.180.190:33249
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25090
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.180.129
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:58:04.655393 25090 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:58:04.656417 25090 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:58:04.666201 25112 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:04.666201 25103 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:58:04.666276 25090 server_base.cc:1034] running on GCE node
W20260811 18:58:04.666260 25102 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:58:04.666828 25090 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:58:04.667491 25090 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:58:04.668656 25090 hybrid_clock.cc:648] HybridClock initialized: now 1786474684668632 us; error 35 us; skew 500 ppm
Aug 11 18:58:04 dist-test-slave-2j7r krb5kdc[24908](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474684, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:58:04.671638 25090 init.cc:377] Logged in from keytab as kudu/127.27.180.129@KRBTEST.COM (short username kudu)
I20260811 18:58:04.672581 25090 webserver.cc:460] Webserver started at http://127.27.180.129:42531/ using document root <none> and password file <none>
I20260811 18:58:04.673192 25090 fs_manager.cc:359] Metadata directory not provided
I20260811 18:58:04.673269 25090 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:58:04.673480 25090 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:58:04.675539 25090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-0/data/instance:
uuid: "b9f8880aef354e3daf773efd8c99ad5c"
format_stamp: "Formatted at 2026-08-11 18:58:04 on dist-test-slave-2j7r"
server_key: "9b6da8953611d63f2da567b5635eb177"
server_key_iv: "3eeb0fea774b51eedc383e0ec2a4fb46"
server_key_version: "encryptionkey@0"
I20260811 18:58:04.676098 25090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-0/wal/instance:
uuid: "b9f8880aef354e3daf773efd8c99ad5c"
format_stamp: "Formatted at 2026-08-11 18:58:04 on dist-test-slave-2j7r"
server_key: "9b6da8953611d63f2da567b5635eb177"
server_key_iv: "3eeb0fea774b51eedc383e0ec2a4fb46"
server_key_version: "encryptionkey@0"
I20260811 18:58:04.680066 25090 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260811 18:58:04.682669 25123 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:04.684075 25090 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 18:58:04.684198 25090 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-0/wal
uuid: "b9f8880aef354e3daf773efd8c99ad5c"
format_stamp: "Formatted at 2026-08-11 18:58:04 on dist-test-slave-2j7r"
server_key: "9b6da8953611d63f2da567b5635eb177"
server_key_iv: "3eeb0fea774b51eedc383e0ec2a4fb46"
server_key_version: "encryptionkey@0"
I20260811 18:58:04.684310 25090 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:58:04.698630 25090 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:58:04.701736 25090 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:58:04.701902 25090 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:58:04.702507 25090 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:58:04.703382 25090 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:58:04.703431 25090 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:04.703485 25090 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:58:04.703512 25090 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:04.712376 25090 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.129:40553
I20260811 18:58:04.712466 25317 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.129:40553 every 8 connection(s)
I20260811 18:58:04.713343 25090 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-0/data/info.pb
I20260811 18:58:04.720901 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 25090
I20260811 18:58:04.721035 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-0/wal/instance
I20260811 18:58:04.721395 28370 external_mini_cluster.cc:1428] Setting key b14782bf1c3bfc15078f4d9f49749b5d
Aug 11 18:58:04 dist-test-slave-2j7r krb5kdc[24908](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474684, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.129@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM
I20260811 18:58:04.726419 25052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:04.715317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.129:53877 (local address 127.27.180.190:33249)
0811 18:58:04.715501 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:04.715505 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:04.716462 (+ 957us) server_negotiation.cc:407] Connection header received
0811 18:58:04.717368 (+ 906us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:04.717371 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:04.717429 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:04.717503 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:04.718654 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:04.719194 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:04.720320 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:04.720552 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:04.720664 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:04.723207 (+ 2543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:58:04.723210 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:58:04.723212 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:58:04.723236 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:58:04.724434 (+ 1198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:04.724901 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:04.724904 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:04.724905 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:04.724992 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:04.725355 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:04.725358 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:04.725360 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:04.725533 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:58:04.725620 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:04.726128 (+ 508us) server_negotiation.cc:299] Negotiation successful
0811 18:58:04.726259 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 18:58:04.727236 25323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:04.715651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33249 (local address 127.27.180.129:53877)
0811 18:58:04.716310 (+ 659us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:04.716352 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:04.717164 (+ 812us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:04.717679 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:04.717685 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:04.718035 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:58:04.718505 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:04.718518 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:04.719315 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:04.719318 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:04.720186 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:04.720195 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:04.720691 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:04.720696 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:04.720869 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:04.721549 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:58:04.721565 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:58:04.723052 (+ 1487us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:58:04.724540 (+ 1488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:58:04.724545 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:58:04.724560 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:58:04.724808 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:58:04.725076 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:58:04.725079 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:58:04.725118 (+ 39us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:58:04.725265 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:58:04.725636 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:58:04.725643 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:58:04.725877 (+ 234us) client_negotiation.cc:769] Sending connection context
0811 18:58:04.726049 (+ 172us) client_negotiation.cc:240] Negotiation successful
0811 18:58:04.726372 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":124,"threads_started":1}
I20260811 18:58:04.728443 25318 heartbeater.cc:347] Connected to a master server at 127.27.180.190:33249
I20260811 18:58:04.728664 25318 heartbeater.cc:464] Registering TS with master...
I20260811 18:58:04.729171 25318 heartbeater.cc:511] Master 127.27.180.190:33249 requested a full tablet report, sending...
I20260811 18:58:04.730803 24982 ts_manager.cc:194] Registered new tserver with Master: b9f8880aef354e3daf773efd8c99ad5c (127.27.180.129:40553)
WARNING: no policy specified for kudu/127.27.180.130@KRBTEST.COM; defaulting to no policy
I20260811 18:58:04.732376 24982 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.180.129@KRBTEST.COM'} at 127.27.180.129:53877
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.180.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.180.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.130.keytab.
Entry for principal kudu/127.27.180.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.130.keytab.
I20260811 18:58:04.751482 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.180.130:0
--local_ip_for_outbound_sockets=127.27.180.130
--webserver_interface=127.27.180.130
--webserver_port=0
--tserver_master_addrs=127.27.180.190:33249
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.130.keytab
--principal=kudu/127.27.180.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:34601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:58:04.865105 25330 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:58:04.865427 25330 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:58:04.865484 25330 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:58:04.871122 25330 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:58:04.871196 25330 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:58:04.871279 25330 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.180.130
I20260811 18:58:04.877249 25330 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:34601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.130.keytab
--principal=kudu/127.27.180.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.180.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.180.130
--webserver_port=0
--tserver_master_addrs=127.27.180.190:33249
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25330
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.180.130
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:58:04.878283 25330 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:58:04.879323 25330 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 18:58:04.888927 25330 server_base.cc:1034] running on GCE node
W20260811 18:58:04.888895 25339 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:04.888895 25336 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:04.889107 25337 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:58:04.889493 25330 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:58:04.890100 25330 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:58:04.891255 25330 hybrid_clock.cc:648] HybridClock initialized: now 1786474684891233 us; error 32 us; skew 500 ppm
Aug 11 18:58:04 dist-test-slave-2j7r krb5kdc[24908](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474684, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:58:04.896863 25330 init.cc:377] Logged in from keytab as kudu/127.27.180.130@KRBTEST.COM (short username kudu)
I20260811 18:58:04.897706 25330 webserver.cc:460] Webserver started at http://127.27.180.130:34561/ using document root <none> and password file <none>
I20260811 18:58:04.898240 25330 fs_manager.cc:359] Metadata directory not provided
I20260811 18:58:04.898305 25330 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:58:04.898502 25330 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:58:04.900738 25330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-1/data/instance:
uuid: "650f778bf77344779339041522b0028a"
format_stamp: "Formatted at 2026-08-11 18:58:04 on dist-test-slave-2j7r"
server_key: "8136d360e419619e8d3e7e7ea272c48a"
server_key_iv: "14844aea4a82af0f1e6fc2d0c734fe6e"
server_key_version: "encryptionkey@0"
I20260811 18:58:04.901333 25330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-1/wal/instance:
uuid: "650f778bf77344779339041522b0028a"
format_stamp: "Formatted at 2026-08-11 18:58:04 on dist-test-slave-2j7r"
server_key: "8136d360e419619e8d3e7e7ea272c48a"
server_key_iv: "14844aea4a82af0f1e6fc2d0c734fe6e"
server_key_version: "encryptionkey@0"
I20260811 18:58:04.905041 25330 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:58:04.907454 25357 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:04.908591 25330 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 18:58:04.908699 25330 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-1/wal
uuid: "650f778bf77344779339041522b0028a"
format_stamp: "Formatted at 2026-08-11 18:58:04 on dist-test-slave-2j7r"
server_key: "8136d360e419619e8d3e7e7ea272c48a"
server_key_iv: "14844aea4a82af0f1e6fc2d0c734fe6e"
server_key_version: "encryptionkey@0"
I20260811 18:58:04.908806 25330 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:58:04.934130 25330 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:58:04.936925 25330 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:58:04.937141 25330 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:58:04.937762 25330 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:58:04.938617 25330 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:58:04.938665 25330 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:04.938706 25330 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:58:04.938726 25330 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:04.947583 25330 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.130:44747
I20260811 18:58:04.947621 25548 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.130:44747 every 8 connection(s)
I20260811 18:58:04.948454 25330 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-1/data/info.pb
I20260811 18:58:04.953684 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 25330
I20260811 18:58:04.953801 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-1/wal/instance
I20260811 18:58:04.954062 28370 external_mini_cluster.cc:1428] Setting key ab1cf94ace334bb4a71454548858eea0
Aug 11 18:58:04 dist-test-slave-2j7r krb5kdc[24908](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474684, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.130@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM
I20260811 18:58:04.961771 25052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:04.950266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.130:44341 (local address 127.27.180.190:33249)
0811 18:58:04.950392 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:04.950396 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:04.951507 (+ 1111us) server_negotiation.cc:407] Connection header received
0811 18:58:04.952428 (+ 921us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:04.952431 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:04.952490 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:04.952572 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:04.953793 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:04.954372 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:04.955692 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:04.955919 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:04.956010 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:04.958483 (+ 2473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:58:04.958486 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:58:04.958488 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:58:04.958513 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:58:04.959777 (+ 1264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:04.960269 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:04.960272 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:04.960274 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:04.960354 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:04.960667 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:04.960670 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:04.960672 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:04.960854 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:58:04.960944 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:04.961477 (+ 533us) server_negotiation.cc:299] Negotiation successful
0811 18:58:04.961603 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260811 18:58:04.962608 25553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:04.950661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33249 (local address 127.27.180.130:44341)
0811 18:58:04.951369 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:04.951412 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:04.952195 (+ 783us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:04.952755 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:04.952762 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:04.953155 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:58:04.953639 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:04.953653 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:04.954502 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:04.954505 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:04.955575 (+ 1070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:04.955586 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:04.956059 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:04.956062 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:04.956222 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:04.956785 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:58:04.956803 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:58:04.958319 (+ 1516us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:58:04.959891 (+ 1572us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:58:04.959897 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:58:04.959912 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:58:04.960165 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:58:04.960438 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:58:04.960441 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:58:04.960443 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:58:04.960579 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:58:04.960954 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:58:04.960961 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:58:04.961245 (+ 284us) client_negotiation.cc:769] Sending connection context
0811 18:58:04.961417 (+ 172us) client_negotiation.cc:240] Negotiation successful
0811 18:58:04.961730 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":127,"threads_started":1}
I20260811 18:58:04.963721 25549 heartbeater.cc:347] Connected to a master server at 127.27.180.190:33249
WARNING: no policy specified for kudu/127.27.180.131@KRBTEST.COM; defaulting to no policy
I20260811 18:58:04.963996 25549 heartbeater.cc:464] Registering TS with master...
I20260811 18:58:04.964480 25549 heartbeater.cc:511] Master 127.27.180.190:33249 requested a full tablet report, sending...
I20260811 18:58:04.965485 24982 ts_manager.cc:194] Registered new tserver with Master: 650f778bf77344779339041522b0028a (127.27.180.130:44747)
I20260811 18:58:04.966110 24982 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.180.130@KRBTEST.COM'} at 127.27.180.130:44341
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.180.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.180.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.131.keytab.
Entry for principal kudu/127.27.180.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.131.keytab.
I20260811 18:58:04.980759 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.180.131:0
--local_ip_for_outbound_sockets=127.27.180.131
--webserver_interface=127.27.180.131
--webserver_port=0
--tserver_master_addrs=127.27.180.190:33249
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.131.keytab
--principal=kudu/127.27.180.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:34601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:58:05.092271 25556 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:58:05.092514 25556 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:58:05.092554 25556 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:58:05.096510 25556 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:58:05.096565 25556 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:58:05.096629 25556 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.180.131
I20260811 18:58:05.101856 25556 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:34601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.131.keytab
--principal=kudu/127.27.180.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.180.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.180.131
--webserver_port=0
--tserver_master_addrs=127.27.180.190:33249
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25556
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.180.131
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:58:05.102905 25556 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:58:05.103943 25556 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:58:05.113219 25569 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:58:05.113257 25556 server_base.cc:1034] running on GCE node
W20260811 18:58:05.113229 25571 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:05.113564 25568 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:58:05.113934 25556 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:58:05.114526 25556 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:58:05.115684 25556 hybrid_clock.cc:648] HybridClock initialized: now 1786474685115669 us; error 33 us; skew 500 ppm
Aug 11 18:58:05 dist-test-slave-2j7r krb5kdc[24908](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474685, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:58:05.118343 25556 init.cc:377] Logged in from keytab as kudu/127.27.180.131@KRBTEST.COM (short username kudu)
I20260811 18:58:05.119166 25556 webserver.cc:460] Webserver started at http://127.27.180.131:40529/ using document root <none> and password file <none>
I20260811 18:58:05.119678 25556 fs_manager.cc:359] Metadata directory not provided
I20260811 18:58:05.119741 25556 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:58:05.119951 25556 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:58:05.121920 25556 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-2/data/instance:
uuid: "d825edd376d9480f8142afe7eefab04f"
format_stamp: "Formatted at 2026-08-11 18:58:05 on dist-test-slave-2j7r"
server_key: "0ae9bcf90361456637956151819b6750"
server_key_iv: "39694be8b7e26463ff6cbc71b959537f"
server_key_version: "encryptionkey@0"
I20260811 18:58:05.122445 25556 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-2/wal/instance:
uuid: "d825edd376d9480f8142afe7eefab04f"
format_stamp: "Formatted at 2026-08-11 18:58:05 on dist-test-slave-2j7r"
server_key: "0ae9bcf90361456637956151819b6750"
server_key_iv: "39694be8b7e26463ff6cbc71b959537f"
server_key_version: "encryptionkey@0"
I20260811 18:58:05.125988 25556 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 18:58:05.128297 25586 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:05.129482 25556 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 18:58:05.129591 25556 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-2/wal
uuid: "d825edd376d9480f8142afe7eefab04f"
format_stamp: "Formatted at 2026-08-11 18:58:05 on dist-test-slave-2j7r"
server_key: "0ae9bcf90361456637956151819b6750"
server_key_iv: "39694be8b7e26463ff6cbc71b959537f"
server_key_version: "encryptionkey@0"
I20260811 18:58:05.129683 25556 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:58:05.142486 25556 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:58:05.145211 25556 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:58:05.145378 25556 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:58:05.145944 25556 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:58:05.146718 25556 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:58:05.146765 25556 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:05.146828 25556 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:58:05.146853 25556 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:05.155503 25556 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.131:44763
I20260811 18:58:05.155542 25781 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.131:44763 every 8 connection(s)
I20260811 18:58:05.156353 25556 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-2/data/info.pb
I20260811 18:58:05.157021 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 25556
I20260811 18:58:05.157155 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-2/wal/instance
I20260811 18:58:05.157426 28370 external_mini_cluster.cc:1428] Setting key 20c396d3294b6f4c1dbf4b7babb14d7a
Aug 11 18:58:05 dist-test-slave-2j7r krb5kdc[24908](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474685, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.131@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM
I20260811 18:58:05.168982 25052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:05.158217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.131:59353 (local address 127.27.180.190:33249)
0811 18:58:05.158333 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:05.158338 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:05.159499 (+ 1161us) server_negotiation.cc:407] Connection header received
0811 18:58:05.160393 (+ 894us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:05.160396 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:05.160443 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:05.160509 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:05.161610 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:05.162128 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:05.163192 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:05.163444 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:05.163534 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:05.165773 (+ 2239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:58:05.165776 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:58:05.165779 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:58:05.165805 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:58:05.167135 (+ 1330us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:05.167566 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:05.167569 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:05.167571 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:05.167655 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:05.167960 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:05.167963 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:05.167965 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:05.168213 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:58:05.168313 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:05.168735 (+ 422us) server_negotiation.cc:299] Negotiation successful
0811 18:58:05.168860 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260811 18:58:05.169730 25788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:05.158641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33249 (local address 127.27.180.131:59353)
0811 18:58:05.159359 (+ 718us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:05.159410 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:05.160203 (+ 793us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:05.160637 (+ 434us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:05.160643 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:05.160986 (+ 343us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:58:05.161478 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:05.161491 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:05.162217 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:05.162219 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:05.163063 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:05.163072 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:05.163565 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:05.163567 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:05.163699 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:05.164209 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:58:05.164225 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:58:05.165618 (+ 1393us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:58:05.167224 (+ 1606us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:58:05.167227 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:58:05.167237 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:58:05.167467 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:58:05.167730 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:58:05.167732 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:58:05.167733 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:58:05.167855 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:58:05.168301 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:58:05.168306 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:58:05.168516 (+ 210us) client_negotiation.cc:769] Sending connection context
0811 18:58:05.168660 (+ 144us) client_negotiation.cc:240] Negotiation successful
0811 18:58:05.168906 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":117,"threads_started":1}
I20260811 18:58:05.170830 25785 heartbeater.cc:347] Connected to a master server at 127.27.180.190:33249
I20260811 18:58:05.171059 25785 heartbeater.cc:464] Registering TS with master...
I20260811 18:58:05.171537 25785 heartbeater.cc:511] Master 127.27.180.190:33249 requested a full tablet report, sending...
I20260811 18:58:05.172488 24982 ts_manager.cc:194] Registered new tserver with Master: d825edd376d9480f8142afe7eefab04f (127.27.180.131:44763)
I20260811 18:58:05.173043 24982 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.180.131@KRBTEST.COM'} at 127.27.180.131:59353
I20260811 18:58:05.182731 28370 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 18:58:05.191336 25052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:05.184637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54014 (local address 127.27.180.190:33249)
0811 18:58:05.184758 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:05.184761 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:05.184927 (+ 166us) server_negotiation.cc:407] Connection header received
0811 18:58:05.185025 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:05.185027 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:05.185065 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:05.185155 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:05.185783 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:05.186303 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:05.187134 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:05.187381 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:05.187457 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:05.188833 (+ 1376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:58:05.188835 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:58:05.188837 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:58:05.188857 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:58:05.190027 (+ 1170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:05.190416 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:05.190418 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:05.190419 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:05.190486 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:05.190660 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:05.190662 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:05.190663 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:05.190856 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:58:05.190958 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:05.191122 (+ 164us) server_negotiation.cc:299] Negotiation successful
0811 18:58:05.191210 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 18:58:05.194056 24982 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54014:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:58:05.196925 24982 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:58:05.211844 25803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:05.205402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.131:44763 (local address 127.0.0.1:41280)
0811 18:58:05.206294 (+ 892us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:05.206347 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:05.206496 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:05.207686 (+ 1190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:05.207689 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:05.207711 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:58:05.207847 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:05.207856 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:05.209382 (+ 1526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:05.209386 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:05.210492 (+ 1106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:05.210501 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:05.211275 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:05.211279 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:05.211407 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:05.211448 (+ 41us) client_negotiation.cc:769] Sending connection context
0811 18:58:05.211552 (+ 104us) client_negotiation.cc:240] Negotiation successful
0811 18:58:05.211682 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":719,"mutex_wait_us":52,"thread_start_us":110,"threads_started":1}
I20260811 18:58:05.212172 25806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:05.205572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41280 (local address 127.27.180.131:44763)
0811 18:58:05.207263 (+ 1691us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:05.207276 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:05.207301 (+ 25us) server_negotiation.cc:407] Connection header received
0811 18:58:05.207380 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:05.207386 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:05.207554 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:05.207714 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:58:05.207970 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:05.209204 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:05.210603 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:05.211160 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:05.211276 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:05.211423 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:05.211929 (+ 506us) server_negotiation.cc:299] Negotiation successful
0811 18:58:05.212037 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1582,"thread_start_us":170,"threads_started":1}
I20260811 18:58:05.212419 25810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:05.205434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.130:44747 (local address 127.0.0.1:41998)
0811 18:58:05.206833 (+ 1399us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:05.206845 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:05.206933 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:05.207454 (+ 521us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:05.207459 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:05.207480 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:58:05.207629 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:05.207638 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:05.208781 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:05.208794 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:05.210156 (+ 1362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:05.210168 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:05.211502 (+ 1334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:05.211506 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:05.211596 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:05.211616 (+ 20us) client_negotiation.cc:769] Sending connection context
0811 18:58:05.211727 (+ 111us) client_negotiation.cc:240] Negotiation successful
0811 18:58:05.211828 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1331,"mutex_wait_us":1,"thread_start_us":116,"threads_started":1}
I20260811 18:58:05.212898 25804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:05.205752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.129:40553 (local address 127.0.0.1:48672)
0811 18:58:05.207099 (+ 1347us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:05.207110 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:05.207192 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:05.207800 (+ 608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:05.207802 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:05.207821 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:58:05.207918 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:05.207923 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:05.209384 (+ 1461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:05.209387 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:05.210309 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:05.210321 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:05.211606 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:05.211611 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:05.211740 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:05.211768 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 18:58:05.211848 (+ 80us) client_negotiation.cc:240] Negotiation successful
0811 18:58:05.211923 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1292,"mutex_wait_us":50,"thread_start_us":98,"threads_started":1}
I20260811 18:58:05.212559 25813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:05.206806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48672 (local address 127.27.180.129:40553)
0811 18:58:05.207127 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:05.207143 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:05.207166 (+ 23us) server_negotiation.cc:407] Connection header received
0811 18:58:05.207307 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:05.207311 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:05.207484 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:05.207635 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:58:05.208034 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:05.209240 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:05.210474 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:05.211303 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:05.211419 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:05.211593 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:05.212201 (+ 608us) server_negotiation.cc:299] Negotiation successful
0811 18:58:05.212396 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":210,"thread_start_us":102,"threads_started":1}
I20260811 18:58:05.212010 25811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:05.206434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41998 (local address 127.27.180.130:44747)
0811 18:58:05.206789 (+ 355us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:05.206804 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:05.206933 (+ 129us) server_negotiation.cc:407] Connection header received
0811 18:58:05.207047 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:05.207053 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:05.207284 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:05.207461 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:58:05.207766 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:05.208621 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:05.210306 (+ 1685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:05.211192 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:05.211335 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:05.211477 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:05.211725 (+ 248us) server_negotiation.cc:299] Negotiation successful
0811 18:58:05.211844 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":238,"thread_start_us":100,"threads_started":1}
I20260811 18:58:05.215657 25438 tablet_service.cc:1467] Processing CreateTablet for tablet ba039a5464b44ac3aa5e7e08f9acc22c (DEFAULT_TABLE table=test-table [id=ae201b4e25a843db854c981be171a570]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:58:05.215842 25673 tablet_service.cc:1467] Processing CreateTablet for tablet ba039a5464b44ac3aa5e7e08f9acc22c (DEFAULT_TABLE table=test-table [id=ae201b4e25a843db854c981be171a570]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:58:05.215842 25207 tablet_service.cc:1467] Processing CreateTablet for tablet ba039a5464b44ac3aa5e7e08f9acc22c (DEFAULT_TABLE table=test-table [id=ae201b4e25a843db854c981be171a570]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:58:05.216791 25438 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba039a5464b44ac3aa5e7e08f9acc22c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:58:05.216885 25207 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba039a5464b44ac3aa5e7e08f9acc22c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:58:05.216902 25673 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba039a5464b44ac3aa5e7e08f9acc22c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:58:05.222993 25814 tablet_bootstrap.cc:492] T ba039a5464b44ac3aa5e7e08f9acc22c P 650f778bf77344779339041522b0028a: Bootstrap starting.
I20260811 18:58:05.224678 25819 tablet_bootstrap.cc:492] T ba039a5464b44ac3aa5e7e08f9acc22c P d825edd376d9480f8142afe7eefab04f: Bootstrap starting.
I20260811 18:58:05.224686 25816 tablet_bootstrap.cc:492] T ba039a5464b44ac3aa5e7e08f9acc22c P b9f8880aef354e3daf773efd8c99ad5c: Bootstrap starting.
I20260811 18:58:05.225829 25814 tablet_bootstrap.cc:654] T ba039a5464b44ac3aa5e7e08f9acc22c P 650f778bf77344779339041522b0028a: Neither blocks nor log segments found. Creating new log.
I20260811 18:58:05.226720 25814 log.cc:824] T ba039a5464b44ac3aa5e7e08f9acc22c P 650f778bf77344779339041522b0028a: Log is configured to *not* fsync() on all Append() calls
I20260811 18:58:05.227042 25816 tablet_bootstrap.cc:654] T ba039a5464b44ac3aa5e7e08f9acc22c P b9f8880aef354e3daf773efd8c99ad5c: Neither blocks nor log segments found. Creating new log.
I20260811 18:58:05.227510 25819 tablet_bootstrap.cc:654] T ba039a5464b44ac3aa5e7e08f9acc22c P d825edd376d9480f8142afe7eefab04f: Neither blocks nor log segments found. Creating new log.
I20260811 18:58:05.228298 25819 log.cc:824] T ba039a5464b44ac3aa5e7e08f9acc22c P d825edd376d9480f8142afe7eefab04f: Log is configured to *not* fsync() on all Append() calls
I20260811 18:58:05.228298 25816 log.cc:824] T ba039a5464b44ac3aa5e7e08f9acc22c P b9f8880aef354e3daf773efd8c99ad5c: Log is configured to *not* fsync() on all Append() calls
I20260811 18:58:05.228773 25814 tablet_bootstrap.cc:492] T ba039a5464b44ac3aa5e7e08f9acc22c P 650f778bf77344779339041522b0028a: No bootstrap required, opened a new log
I20260811 18:58:05.228984 25814 ts_tablet_manager.cc:1397] T ba039a5464b44ac3aa5e7e08f9acc22c P 650f778bf77344779339041522b0028a: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 18:58:05.230299 25819 tablet_bootstrap.cc:492] T ba039a5464b44ac3aa5e7e08f9acc22c P d825edd376d9480f8142afe7eefab04f: No bootstrap required, opened a new log
I20260811 18:58:05.230296 25816 tablet_bootstrap.cc:492] T ba039a5464b44ac3aa5e7e08f9acc22c P b9f8880aef354e3daf773efd8c99ad5c: No bootstrap required, opened a new log
I20260811 18:58:05.230494 25816 ts_tablet_manager.cc:1397] T ba039a5464b44ac3aa5e7e08f9acc22c P b9f8880aef354e3daf773efd8c99ad5c: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 18:58:05.230494 25819 ts_tablet_manager.cc:1397] T ba039a5464b44ac3aa5e7e08f9acc22c P d825edd376d9480f8142afe7eefab04f: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.001s
I20260811 18:58:05.231920 25814 raft_consensus.cc:348] T ba039a5464b44ac3aa5e7e08f9acc22c P 650f778bf77344779339041522b0028a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d825edd376d9480f8142afe7eefab04f" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 44763 } } peers { permanent_uuid: "650f778bf77344779339041522b0028a" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 44747 } } peers { permanent_uuid: "b9f8880aef354e3daf773efd8c99ad5c" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 40553 } }
I20260811 18:58:05.232149 25814 raft_consensus.cc:374] T ba039a5464b44ac3aa5e7e08f9acc22c P 650f778bf77344779339041522b0028a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:58:05.232204 25814 raft_consensus.cc:729] T ba039a5464b44ac3aa5e7e08f9acc22c P 650f778bf77344779339041522b0028a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 650f778bf77344779339041522b0028a, State: Initialized, Role: FOLLOWER
I20260811 18:58:05.232664 25814 consensus_queue.cc:260] T ba039a5464b44ac3aa5e7e08f9acc22c P 650f778bf77344779339041522b0028a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d825edd376d9480f8142afe7eefab04f" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 44763 } } peers { permanent_uuid: "650f778bf77344779339041522b0028a" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 44747 } } peers { permanent_uuid: "b9f8880aef354e3daf773efd8c99ad5c" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 40553 } }
I20260811 18:58:05.233310 25549 heartbeater.cc:503] Master 127.27.180.190:33249 was elected leader, sending a full tablet report...
I20260811 18:58:05.234512 25814 ts_tablet_manager.cc:1428] T ba039a5464b44ac3aa5e7e08f9acc22c P 650f778bf77344779339041522b0028a: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260811 18:58:05.234483 25819 raft_consensus.cc:348] T ba039a5464b44ac3aa5e7e08f9acc22c P d825edd376d9480f8142afe7eefab04f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d825edd376d9480f8142afe7eefab04f" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 44763 } } peers { permanent_uuid: "650f778bf77344779339041522b0028a" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 44747 } } peers { permanent_uuid: "b9f8880aef354e3daf773efd8c99ad5c" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 40553 } }
I20260811 18:58:05.234483 25816 raft_consensus.cc:348] T ba039a5464b44ac3aa5e7e08f9acc22c P b9f8880aef354e3daf773efd8c99ad5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d825edd376d9480f8142afe7eefab04f" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 44763 } } peers { permanent_uuid: "650f778bf77344779339041522b0028a" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 44747 } } peers { permanent_uuid: "b9f8880aef354e3daf773efd8c99ad5c" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 40553 } }
I20260811 18:58:05.234760 25816 raft_consensus.cc:374] T ba039a5464b44ac3aa5e7e08f9acc22c P b9f8880aef354e3daf773efd8c99ad5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:58:05.234760 25819 raft_consensus.cc:374] T ba039a5464b44ac3aa5e7e08f9acc22c P d825edd376d9480f8142afe7eefab04f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:58:05.234826 25819 raft_consensus.cc:729] T ba039a5464b44ac3aa5e7e08f9acc22c P d825edd376d9480f8142afe7eefab04f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d825edd376d9480f8142afe7eefab04f, State: Initialized, Role: FOLLOWER
I20260811 18:58:05.234826 25816 raft_consensus.cc:729] T ba039a5464b44ac3aa5e7e08f9acc22c P b9f8880aef354e3daf773efd8c99ad5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9f8880aef354e3daf773efd8c99ad5c, State: Initialized, Role: FOLLOWER
I20260811 18:58:05.235262 25816 consensus_queue.cc:260] T ba039a5464b44ac3aa5e7e08f9acc22c P b9f8880aef354e3daf773efd8c99ad5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d825edd376d9480f8142afe7eefab04f" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 44763 } } peers { permanent_uuid: "650f778bf77344779339041522b0028a" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 44747 } } peers { permanent_uuid: "b9f8880aef354e3daf773efd8c99ad5c" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 40553 } }
I20260811 18:58:05.235311 25819 consensus_queue.cc:260] T ba039a5464b44ac3aa5e7e08f9acc22c P d825edd376d9480f8142afe7eefab04f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d825edd376d9480f8142afe7eefab04f" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 44763 } } peers { permanent_uuid: "650f778bf77344779339041522b0028a" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 44747 } } peers { permanent_uuid: "b9f8880aef354e3daf773efd8c99ad5c" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 40553 } }
I20260811 18:58:05.236166 25318 heartbeater.cc:503] Master 127.27.180.190:33249 was elected leader, sending a full tablet report...
I20260811 18:58:05.236272 25816 ts_tablet_manager.cc:1428] T ba039a5464b44ac3aa5e7e08f9acc22c P b9f8880aef354e3daf773efd8c99ad5c: Time spent starting tablet: real 0.006s user 0.003s sys 0.003s
I20260811 18:58:05.236272 25819 ts_tablet_manager.cc:1428] T ba039a5464b44ac3aa5e7e08f9acc22c P d825edd376d9480f8142afe7eefab04f: Time spent starting tablet: real 0.006s user 0.003s sys 0.002s
I20260811 18:58:05.237174 25785 heartbeater.cc:503] Master 127.27.180.190:33249 was elected leader, sending a full tablet report...
W20260811 18:58:05.407323 25787 tablet.cc:2369] T ba039a5464b44ac3aa5e7e08f9acc22c P d825edd376d9480f8142afe7eefab04f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 18:58:05.449630 25552 tablet.cc:2369] T ba039a5464b44ac3aa5e7e08f9acc22c P 650f778bf77344779339041522b0028a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:58:05.463959 25831 raft_consensus.cc:482] T ba039a5464b44ac3aa5e7e08f9acc22c P d825edd376d9480f8142afe7eefab04f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:58:05.464154 25831 raft_consensus.cc:504] T ba039a5464b44ac3aa5e7e08f9acc22c P d825edd376d9480f8142afe7eefab04f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d825edd376d9480f8142afe7eefab04f" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 44763 } } peers { permanent_uuid: "650f778bf77344779339041522b0028a" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 44747 } } peers { permanent_uuid: "b9f8880aef354e3daf773efd8c99ad5c" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 40553 } }
I20260811 18:58:05.465373 25831 leader_election.cc:288] T ba039a5464b44ac3aa5e7e08f9acc22c P d825edd376d9480f8142afe7eefab04f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 650f778bf77344779339041522b0028a (127.27.180.130:44747), b9f8880aef354e3daf773efd8c99ad5c (127.27.180.129:40553)
W20260811 18:58:05.466668 25320 tablet.cc:2369] T ba039a5464b44ac3aa5e7e08f9acc22c P b9f8880aef354e3daf773efd8c99ad5c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:58:05.469633 25811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:05.465728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.131:33549 (local address 127.27.180.130:44747)
0811 18:58:05.465859 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:05.465865 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:05.465885 (+ 20us) server_negotiation.cc:407] Connection header received
0811 18:58:05.466041 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:05.466044 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:05.466090 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:05.466166 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:58:05.466491 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:05.467137 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:05.468469 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:05.468946 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:05.469061 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:05.469142 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:05.469447 (+ 305us) server_negotiation.cc:299] Negotiation successful
0811 18:58:05.469529 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 18:58:05.469636 25788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:05.465656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.130:44747 (local address 127.27.180.131:33549)
0811 18:58:05.465803 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:05.465819 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:05.465937 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:05.466195 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:05.466198 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:05.466218 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:58:05.466345 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:05.466352 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:05.467238 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:05.467241 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:05.468331 (+ 1090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:05.468343 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:05.469177 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:05.469180 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:05.469301 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:05.469336 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 18:58:05.469417 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 18:58:05.469495 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260811 18:58:05.469841 25832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:05.465671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.129:40553 (local address 127.27.180.131:52423)
0811 18:58:05.466282 (+ 611us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:05.466294 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:05.466379 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:05.466619 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:05.466622 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:05.466640 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:58:05.466777 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:05.466784 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:05.467635 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:05.467639 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:05.468645 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:05.468653 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:05.469457 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:05.469460 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:05.469550 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:05.469574 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 18:58:05.469644 (+ 70us) client_negotiation.cc:240] Negotiation successful
0811 18:58:05.469716 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":547,"mutex_wait_us":331,"thread_start_us":94,"threads_started":1}
I20260811 18:58:05.469861 25813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:05.465807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.131:52423 (local address 127.27.180.129:40553)
0811 18:58:05.465964 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:05.465967 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:05.466371 (+ 404us) server_negotiation.cc:407] Connection header received
0811 18:58:05.466454 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:05.466457 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:05.466528 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:05.466602 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:58:05.466923 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:05.467515 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:05.468763 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:05.469340 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:05.469445 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:05.469495 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:05.469672 (+ 177us) server_negotiation.cc:299] Negotiation successful
0811 18:58:05.469744 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 18:58:05.470405 25238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba039a5464b44ac3aa5e7e08f9acc22c" candidate_uuid: "d825edd376d9480f8142afe7eefab04f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9f8880aef354e3daf773efd8c99ad5c" is_pre_election: true
I20260811 18:58:05.470546 25475 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba039a5464b44ac3aa5e7e08f9acc22c" candidate_uuid: "d825edd376d9480f8142afe7eefab04f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "650f778bf77344779339041522b0028a" is_pre_election: true
I20260811 18:58:05.470674 25238 raft_consensus.cc:2454] T ba039a5464b44ac3aa5e7e08f9acc22c P b9f8880aef354e3daf773efd8c99ad5c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d825edd376d9480f8142afe7eefab04f in term 0.
I20260811 18:58:05.470808 25475 raft_consensus.cc:2454] T ba039a5464b44ac3aa5e7e08f9acc22c P 650f778bf77344779339041522b0028a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d825edd376d9480f8142afe7eefab04f in term 0.
I20260811 18:58:05.471155 25592 leader_election.cc:302] T ba039a5464b44ac3aa5e7e08f9acc22c P d825edd376d9480f8142afe7eefab04f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b9f8880aef354e3daf773efd8c99ad5c, d825edd376d9480f8142afe7eefab04f; no voters:
I20260811 18:58:05.471457 25831 raft_consensus.cc:2781] T ba039a5464b44ac3aa5e7e08f9acc22c P d825edd376d9480f8142afe7eefab04f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:58:05.471539 25831 raft_consensus.cc:482] T ba039a5464b44ac3aa5e7e08f9acc22c P d825edd376d9480f8142afe7eefab04f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:58:05.471588 25831 raft_consensus.cc:3037] T ba039a5464b44ac3aa5e7e08f9acc22c P d825edd376d9480f8142afe7eefab04f [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:58:05.472643 25831 raft_consensus.cc:504] T ba039a5464b44ac3aa5e7e08f9acc22c P d825edd376d9480f8142afe7eefab04f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d825edd376d9480f8142afe7eefab04f" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 44763 } } peers { permanent_uuid: "650f778bf77344779339041522b0028a" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 44747 } } peers { permanent_uuid: "b9f8880aef354e3daf773efd8c99ad5c" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 40553 } }
I20260811 18:58:05.472987 25831 leader_election.cc:288] T ba039a5464b44ac3aa5e7e08f9acc22c P d825edd376d9480f8142afe7eefab04f [CANDIDATE]: Term 1 election: Requested vote from peers 650f778bf77344779339041522b0028a (127.27.180.130:44747), b9f8880aef354e3daf773efd8c99ad5c (127.27.180.129:40553)
I20260811 18:58:05.473359 25475 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba039a5464b44ac3aa5e7e08f9acc22c" candidate_uuid: "d825edd376d9480f8142afe7eefab04f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "650f778bf77344779339041522b0028a"
I20260811 18:58:05.473384 25238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba039a5464b44ac3aa5e7e08f9acc22c" candidate_uuid: "d825edd376d9480f8142afe7eefab04f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9f8880aef354e3daf773efd8c99ad5c"
I20260811 18:58:05.473485 25238 raft_consensus.cc:3037] T ba039a5464b44ac3aa5e7e08f9acc22c P b9f8880aef354e3daf773efd8c99ad5c [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:58:05.473486 25475 raft_consensus.cc:3037] T ba039a5464b44ac3aa5e7e08f9acc22c P 650f778bf77344779339041522b0028a [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:58:05.474740 25475 raft_consensus.cc:2454] T ba039a5464b44ac3aa5e7e08f9acc22c P 650f778bf77344779339041522b0028a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d825edd376d9480f8142afe7eefab04f in term 1.
I20260811 18:58:05.474735 25238 raft_consensus.cc:2454] T ba039a5464b44ac3aa5e7e08f9acc22c P b9f8880aef354e3daf773efd8c99ad5c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d825edd376d9480f8142afe7eefab04f in term 1.
I20260811 18:58:05.475033 25592 leader_election.cc:302] T ba039a5464b44ac3aa5e7e08f9acc22c P d825edd376d9480f8142afe7eefab04f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b9f8880aef354e3daf773efd8c99ad5c, d825edd376d9480f8142afe7eefab04f; no voters:
I20260811 18:58:05.475189 25831 raft_consensus.cc:2781] T ba039a5464b44ac3aa5e7e08f9acc22c P d825edd376d9480f8142afe7eefab04f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:58:05.475449 25831 raft_consensus.cc:686] T ba039a5464b44ac3aa5e7e08f9acc22c P d825edd376d9480f8142afe7eefab04f [term 1 LEADER]: Becoming Leader. State: Replica: d825edd376d9480f8142afe7eefab04f, State: Running, Role: LEADER
I20260811 18:58:05.475780 25831 consensus_queue.cc:237] T ba039a5464b44ac3aa5e7e08f9acc22c P d825edd376d9480f8142afe7eefab04f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d825edd376d9480f8142afe7eefab04f" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 44763 } } peers { permanent_uuid: "650f778bf77344779339041522b0028a" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 44747 } } peers { permanent_uuid: "b9f8880aef354e3daf773efd8c99ad5c" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 40553 } }
I20260811 18:58:05.479092 24987 catalog_manager.cc:5495] T ba039a5464b44ac3aa5e7e08f9acc22c P d825edd376d9480f8142afe7eefab04f reported cstate change: term changed from 0 to 1, leader changed from <none> to d825edd376d9480f8142afe7eefab04f (127.27.180.131). New cstate: current_term: 1 leader_uuid: "d825edd376d9480f8142afe7eefab04f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d825edd376d9480f8142afe7eefab04f" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 44763 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "650f778bf77344779339041522b0028a" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 44747 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b9f8880aef354e3daf773efd8c99ad5c" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 40553 } health_report { overall_health: UNKNOWN } } }
I20260811 18:58:05.502817 25806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:05.499188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41294 (local address 127.27.180.131:44763)
0811 18:58:05.499334 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:05.499338 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:05.499354 (+ 16us) server_negotiation.cc:407] Connection header received
0811 18:58:05.499498 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:05.499500 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:05.499559 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:05.499650 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:58:05.499919 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:05.500422 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:05.501331 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:05.501560 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:05.501651 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:05.501973 (+ 322us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:58:05.502415 (+ 442us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:58:05.502525 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:05.502603 (+ 78us) server_negotiation.cc:299] Negotiation successful
0811 18:58:05.502666 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 18:58:05.508667 25475 raft_consensus.cc:1261] T ba039a5464b44ac3aa5e7e08f9acc22c P 650f778bf77344779339041522b0028a [term 1 FOLLOWER]: Refusing update from remote peer d825edd376d9480f8142afe7eefab04f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:58:05.508668 25238 raft_consensus.cc:1261] T ba039a5464b44ac3aa5e7e08f9acc22c P b9f8880aef354e3daf773efd8c99ad5c [term 1 FOLLOWER]: Refusing update from remote peer d825edd376d9480f8142afe7eefab04f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:58:05.509374 25833 consensus_queue.cc:1035] T ba039a5464b44ac3aa5e7e08f9acc22c P d825edd376d9480f8142afe7eefab04f [LEADER]: Connected to new peer: Peer: permanent_uuid: "650f778bf77344779339041522b0028a" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 44747 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:58:05.509549 25831 consensus_queue.cc:1035] T ba039a5464b44ac3aa5e7e08f9acc22c P d825edd376d9480f8142afe7eefab04f [LEADER]: Connected to new peer: Peer: permanent_uuid: "b9f8880aef354e3daf773efd8c99ad5c" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 40553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:58:05.516393 25837 mvcc.cc:205] Tried to move back new op lower bound from 7317400311836479488 to 7317400311711289344. Current Snapshot: MvccSnapshot[applied={T|T < 7317400311836479488}]
I20260811 18:58:05.517211 25839 mvcc.cc:205] Tried to move back new op lower bound from 7317400311836479488 to 7317400311711289344. Current Snapshot: MvccSnapshot[applied={T|T < 7317400311836479488}]
I20260811 18:58:05.519702 25841 mvcc.cc:205] Tried to move back new op lower bound from 7317400311836479488 to 7317400311711289344. Current Snapshot: MvccSnapshot[applied={T|T < 7317400311836479488}]
I20260811 18:58:05.525252 24987 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54014:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:58:05.525444 24987 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54014:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:58:05.528177 24987 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b3054cf295fb43d6bb0787856013df0e: Sending DeleteTablet for 3 replicas of tablet ba039a5464b44ac3aa5e7e08f9acc22c
I20260811 18:58:05.529037 25673 tablet_service.cc:1514] Processing DeleteTablet for tablet ba039a5464b44ac3aa5e7e08f9acc22c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:58:05 UTC) from {username='kudu', principal='kudu/127.27.180.190@KRBTEST.COM'} at 127.0.0.1:41280
I20260811 18:58:05.529052 25438 tablet_service.cc:1514] Processing DeleteTablet for tablet ba039a5464b44ac3aa5e7e08f9acc22c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:58:05 UTC) from {username='kudu', principal='kudu/127.27.180.190@KRBTEST.COM'} at 127.0.0.1:41998
I20260811 18:58:05.529258 25207 tablet_service.cc:1514] Processing DeleteTablet for tablet ba039a5464b44ac3aa5e7e08f9acc22c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:58:05 UTC) from {username='kudu', principal='kudu/127.27.180.190@KRBTEST.COM'} at 127.0.0.1:48672
I20260811 18:58:05.529430 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 25090
I20260811 18:58:05.550088 25856 tablet_replica.cc:330] T ba039a5464b44ac3aa5e7e08f9acc22c P 650f778bf77344779339041522b0028a: stopping tablet replica
I20260811 18:58:05.550383 25858 tablet_replica.cc:330] T ba039a5464b44ac3aa5e7e08f9acc22c P d825edd376d9480f8142afe7eefab04f: stopping tablet replica
I20260811 18:58:05.550642 25856 raft_consensus.cc:2229] T ba039a5464b44ac3aa5e7e08f9acc22c P 650f778bf77344779339041522b0028a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:58:05.550774 25858 raft_consensus.cc:2229] T ba039a5464b44ac3aa5e7e08f9acc22c P d825edd376d9480f8142afe7eefab04f [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:58:05.551206 25856 raft_consensus.cc:2258] T ba039a5464b44ac3aa5e7e08f9acc22c P 650f778bf77344779339041522b0028a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:58:05.551273 25858 raft_consensus.cc:2258] T ba039a5464b44ac3aa5e7e08f9acc22c P d825edd376d9480f8142afe7eefab04f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:58:05.552987 25858 ts_tablet_manager.cc:1905] T ba039a5464b44ac3aa5e7e08f9acc22c P d825edd376d9480f8142afe7eefab04f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:58:05.553562 25856 ts_tablet_manager.cc:1905] T ba039a5464b44ac3aa5e7e08f9acc22c P 650f778bf77344779339041522b0028a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:58:05.555985 25858 ts_tablet_manager.cc:1918] T ba039a5464b44ac3aa5e7e08f9acc22c P d825edd376d9480f8142afe7eefab04f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:58:05.556084 25858 log.cc:1196] T ba039a5464b44ac3aa5e7e08f9acc22c P d825edd376d9480f8142afe7eefab04f: Deleting WAL directory at /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-2/wal/wals/ba039a5464b44ac3aa5e7e08f9acc22c
I20260811 18:58:05.556423 25858 ts_tablet_manager.cc:1939] T ba039a5464b44ac3aa5e7e08f9acc22c P d825edd376d9480f8142afe7eefab04f: Deleting consensus metadata
I20260811 18:58:05.556586 25856 ts_tablet_manager.cc:1918] T ba039a5464b44ac3aa5e7e08f9acc22c P 650f778bf77344779339041522b0028a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:58:05.556734 25856 log.cc:1196] T ba039a5464b44ac3aa5e7e08f9acc22c P 650f778bf77344779339041522b0028a: Deleting WAL directory at /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-1/wal/wals/ba039a5464b44ac3aa5e7e08f9acc22c
I20260811 18:58:05.557139 25856 ts_tablet_manager.cc:1939] T ba039a5464b44ac3aa5e7e08f9acc22c P 650f778bf77344779339041522b0028a: Deleting consensus metadata
I20260811 18:58:05.557746 24955 catalog_manager.cc:4841] TS d825edd376d9480f8142afe7eefab04f (127.27.180.131:44763): tablet ba039a5464b44ac3aa5e7e08f9acc22c (table test-table [id=ae201b4e25a843db854c981be171a570]) successfully deleted
W20260811 18:58:05.558213 24957 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.27.180.129:40553 (error 108)
I20260811 18:58:05.558386 24960 catalog_manager.cc:4841] TS 650f778bf77344779339041522b0028a (127.27.180.130:44747): tablet ba039a5464b44ac3aa5e7e08f9acc22c (table test-table [id=ae201b4e25a843db854c981be171a570]) successfully deleted
I20260811 18:58:05.558589 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 25330
I20260811 18:58:05.563616 25804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:05.563286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.129:40553 (local address 127.0.0.1:48682)
0811 18:58:05.563467 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:05.563533 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.129:40553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110}
W20260811 18:58:05.563791 24957 catalog_manager.cc:4584] TS b9f8880aef354e3daf773efd8c99ad5c (127.27.180.129:40553): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ba039a5464b44ac3aa5e7e08f9acc22c: Network error: Client connection negotiation failed: client connection to 127.27.180.129:40553: connect: Connection refused (error 111)
I20260811 18:58:05.578974 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 25556
I20260811 18:58:05.590698 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 24931
I20260811 18:58:25.601817 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.190:33249
--webserver_interface=127.27.180.190
--webserver_port=33175
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab
--principal=kudu/127.27.180.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:34601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.180.190:33249
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:58:25.717590 25860 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:58:25.717892 25860 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:58:25.717959 25860 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:58:25.721982 25860 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 18:58:25.722041 25860 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 18:58:25.722067 25860 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 18:58:25.722088 25860 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:58:25.722118 25860 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:58:25.722154 25860 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:58:25.722200 25860 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:58:25.722229 25860 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:58:25.727579 25860 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:34601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.27.180.190:33249
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab
--principal=kudu/127.27.180.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.190:33249
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.180.190
--webserver_port=33175
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25860
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:58:25.728720 25860 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:58:25.729789 25860 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:58:25.743964 25875 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:25.743964 25872 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:25.744001 25870 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:58:25.744190 25860 server_base.cc:1034] running on GCE node
I20260811 18:58:25.744858 25860 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:58:25.746052 25860 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:58:25.747206 25860 hybrid_clock.cc:648] HybridClock initialized: now 1786474705747192 us; error 28 us; skew 500 ppm
Aug 11 18:58:25 dist-test-slave-2j7r krb5kdc[24908](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474705, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:58:25.750113 25860 init.cc:377] Logged in from keytab as kudu/127.27.180.190@KRBTEST.COM (short username kudu)
I20260811 18:58:25.751036 25860 webserver.cc:460] Webserver started at http://127.27.180.190:33175/ using document root <none> and password file <none>
I20260811 18:58:25.751600 25860 fs_manager.cc:359] Metadata directory not provided
I20260811 18:58:25.751685 25860 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:58:25.755604 25860 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260811 18:58:25.757695 25885 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:25.758921 25860 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 18:58:25.759038 25860 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/master-0/wal
uuid: "b3054cf295fb43d6bb0787856013df0e"
format_stamp: "Formatted at 2026-08-11 18:58:04 on dist-test-slave-2j7r"
server_key: "7cb1e0c8cf224b542b7cef8b2d22b75d"
server_key_iv: "4b3cfb7012c4f74187c871d278b3bb3b"
server_key_version: "encryptionkey@0"
I20260811 18:58:25.759644 25860 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:58:25.776662 25860 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:58:25.779807 25860 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:58:25.779999 25860 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:58:25.787271 25860 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.190:33249
I20260811 18:58:25.787570 25972 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.190:33249 every 8 connection(s)
I20260811 18:58:25.788360 25860 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
I20260811 18:58:25.790103 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 25860
I20260811 18:58:25.790511 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.180.129:40553
--local_ip_for_outbound_sockets=127.27.180.129
--tserver_master_addrs=127.27.180.190:33249
--webserver_port=42531
--webserver_interface=127.27.180.129
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.129.keytab
--principal=kudu/127.27.180.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:34601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 18:58:25.792878 25975 sys_catalog.cc:263] Verifying existing consensus state
I20260811 18:58:25.794576 25975 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b3054cf295fb43d6bb0787856013df0e: Bootstrap starting.
I20260811 18:58:25.812495 25975 log.cc:824] T 00000000000000000000000000000000 P b3054cf295fb43d6bb0787856013df0e: Log is configured to *not* fsync() on all Append() calls
I20260811 18:58:25.821421 25975 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b3054cf295fb43d6bb0787856013df0e: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 18:58:25.821765 25975 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b3054cf295fb43d6bb0787856013df0e: Bootstrap complete.
I20260811 18:58:25.825763 25975 raft_consensus.cc:348] T 00000000000000000000000000000000 P b3054cf295fb43d6bb0787856013df0e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3054cf295fb43d6bb0787856013df0e" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 33249 } }
I20260811 18:58:25.826191 25975 raft_consensus.cc:729] T 00000000000000000000000000000000 P b3054cf295fb43d6bb0787856013df0e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3054cf295fb43d6bb0787856013df0e, State: Initialized, Role: FOLLOWER
I20260811 18:58:25.826684 25975 consensus_queue.cc:260] T 00000000000000000000000000000000 P b3054cf295fb43d6bb0787856013df0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3054cf295fb43d6bb0787856013df0e" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 33249 } }
I20260811 18:58:25.826814 25975 raft_consensus.cc:388] T 00000000000000000000000000000000 P b3054cf295fb43d6bb0787856013df0e [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:58:25.826890 25975 raft_consensus.cc:482] T 00000000000000000000000000000000 P b3054cf295fb43d6bb0787856013df0e [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:58:25.826989 25975 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b3054cf295fb43d6bb0787856013df0e [term 1 FOLLOWER]: Advancing to term 2
I20260811 18:58:25.828941 25975 raft_consensus.cc:504] T 00000000000000000000000000000000 P b3054cf295fb43d6bb0787856013df0e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3054cf295fb43d6bb0787856013df0e" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 33249 } }
I20260811 18:58:25.829276 25975 leader_election.cc:302] T 00000000000000000000000000000000 P b3054cf295fb43d6bb0787856013df0e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b3054cf295fb43d6bb0787856013df0e; no voters:
I20260811 18:58:25.829586 25975 leader_election.cc:288] T 00000000000000000000000000000000 P b3054cf295fb43d6bb0787856013df0e [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 18:58:25.829694 25983 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b3054cf295fb43d6bb0787856013df0e [term 2 FOLLOWER]: Leader election won for term 2
I20260811 18:58:25.829905 25983 raft_consensus.cc:686] T 00000000000000000000000000000000 P b3054cf295fb43d6bb0787856013df0e [term 2 LEADER]: Becoming Leader. State: Replica: b3054cf295fb43d6bb0787856013df0e, State: Running, Role: LEADER
I20260811 18:58:25.830263 25983 consensus_queue.cc:237] T 00000000000000000000000000000000 P b3054cf295fb43d6bb0787856013df0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3054cf295fb43d6bb0787856013df0e" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 33249 } }
I20260811 18:58:25.830860 25975 sys_catalog.cc:564] T 00000000000000000000000000000000 P b3054cf295fb43d6bb0787856013df0e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:58:25.831283 25983 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3054cf295fb43d6bb0787856013df0e [sys.catalog]: SysCatalogTable state changed. Reason: New leader b3054cf295fb43d6bb0787856013df0e. Latest consensus state: current_term: 2 leader_uuid: "b3054cf295fb43d6bb0787856013df0e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3054cf295fb43d6bb0787856013df0e" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 33249 } } }
I20260811 18:58:25.831485 25983 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3054cf295fb43d6bb0787856013df0e [sys.catalog]: This master's current role is: LEADER
I20260811 18:58:25.831696 25984 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3054cf295fb43d6bb0787856013df0e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "b3054cf295fb43d6bb0787856013df0e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3054cf295fb43d6bb0787856013df0e" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 33249 } } }
I20260811 18:58:25.831919 25995 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:58:25.831902 25984 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3054cf295fb43d6bb0787856013df0e [sys.catalog]: This master's current role is: LEADER
I20260811 18:58:25.838783 25995 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:58:25.839645 25995 catalog_manager.cc:1259] Loaded cluster ID: a07c061746b84cfda0f8ef5a97059df7
I20260811 18:58:25.839859 25995 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:58:25.842324 25995 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:58:25.843053 25995 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b3054cf295fb43d6bb0787856013df0e: Loaded TSK: 0
I20260811 18:58:25.862268 25995 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b3054cf295fb43d6bb0787856013df0e: Generated new TSK 1
I20260811 18:58:25.862545 25995 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 18:58:25.912319 25978 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:58:25.912575 25978 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:58:25.912635 25978 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:58:25.916571 25978 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:58:25.916627 25978 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:58:25.916705 25978 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.180.129
I20260811 18:58:25.922020 25978 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:34601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.129.keytab
--principal=kudu/127.27.180.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.180.129:40553
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.180.129
--webserver_port=42531
--tserver_master_addrs=127.27.180.190:33249
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25978
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.180.129
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:58:25.923116 25978 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:58:25.924103 25978 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:58:25.934863 26026 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:25.934958 26025 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:25.934857 26032 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:58:25.935333 25978 server_base.cc:1034] running on GCE node
I20260811 18:58:25.935734 25978 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:58:25.936374 25978 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:58:25.937559 25978 hybrid_clock.cc:648] HybridClock initialized: now 1786474705937529 us; error 48 us; skew 500 ppm
Aug 11 18:58:25 dist-test-slave-2j7r krb5kdc[24908](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474705, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:58:25.940339 25978 init.cc:377] Logged in from keytab as kudu/127.27.180.129@KRBTEST.COM (short username kudu)
I20260811 18:58:25.941331 25978 webserver.cc:460] Webserver started at http://127.27.180.129:42531/ using document root <none> and password file <none>
I20260811 18:58:25.941906 25978 fs_manager.cc:359] Metadata directory not provided
I20260811 18:58:25.941992 25978 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:58:25.946056 25978 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 18:58:25.948773 26041 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:25.950151 25978 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 18:58:25.950307 25978 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-0/wal
uuid: "b9f8880aef354e3daf773efd8c99ad5c"
format_stamp: "Formatted at 2026-08-11 18:58:04 on dist-test-slave-2j7r"
server_key: "9b6da8953611d63f2da567b5635eb177"
server_key_iv: "3eeb0fea774b51eedc383e0ec2a4fb46"
server_key_version: "encryptionkey@0"
I20260811 18:58:25.950872 25978 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:58:25.983855 25978 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:58:25.987210 25978 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:58:25.987416 25978 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:58:25.988072 25978 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:58:25.989239 26051 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 18:58:25.993419 25978 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 18:58:25.993492 25978 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.000s sys 0.001s
I20260811 18:58:25.993566 25978 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 18:58:25.995236 25978 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 18:58:25.995282 25978 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260811 18:58:25.995571 26051 tablet_bootstrap.cc:492] T ba039a5464b44ac3aa5e7e08f9acc22c P b9f8880aef354e3daf773efd8c99ad5c: Bootstrap starting.
I20260811 18:58:26.004839 25978 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.129:40553
I20260811 18:58:26.004905 26228 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.129:40553 every 8 connection(s)
I20260811 18:58:26.005796 25978 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-0/data/info.pb
I20260811 18:58:26.009912 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 25978
I20260811 18:58:26.010408 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.180.130:44747
--local_ip_for_outbound_sockets=127.27.180.130
--tserver_master_addrs=127.27.180.190:33249
--webserver_port=34561
--webserver_interface=127.27.180.130
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.130.keytab
--principal=kudu/127.27.180.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:34601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 18:58:26.015220 26051 log.cc:824] T ba039a5464b44ac3aa5e7e08f9acc22c P b9f8880aef354e3daf773efd8c99ad5c: Log is configured to *not* fsync() on all Append() calls
Aug 11 18:58:26 dist-test-slave-2j7r krb5kdc[24908](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474705, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.129@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM
I20260811 18:58:26.023219 26051 tablet_bootstrap.cc:492] T ba039a5464b44ac3aa5e7e08f9acc22c P b9f8880aef354e3daf773efd8c99ad5c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 18:58:26.023600 26051 tablet_bootstrap.cc:492] T ba039a5464b44ac3aa5e7e08f9acc22c P b9f8880aef354e3daf773efd8c99ad5c: Bootstrap complete.
I20260811 18:58:26.024613 26051 ts_tablet_manager.cc:1397] T ba039a5464b44ac3aa5e7e08f9acc22c P b9f8880aef354e3daf773efd8c99ad5c: Time spent bootstrapping tablet: real 0.029s user 0.025s sys 0.000s
I20260811 18:58:26.025309 26235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:26.007697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33249 (local address 127.27.180.129:53889)
0811 18:58:26.008293 (+ 596us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:26.008362 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:26.009492 (+ 1130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:26.010910 (+ 1418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:26.010919 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:26.011444 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:58:26.012147 (+ 703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:26.012168 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:26.013993 (+ 1825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:26.013996 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:26.015267 (+ 1271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:26.015278 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:26.016206 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:26.016211 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:26.016399 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:26.017072 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:58:26.017117 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:58:26.019012 (+ 1895us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:58:26.021545 (+ 2533us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:58:26.021551 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:58:26.021566 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:58:26.021953 (+ 387us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:58:26.022412 (+ 459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:58:26.022415 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:58:26.022417 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:58:26.022593 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:58:26.023365 (+ 772us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:58:26.023372 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:58:26.023681 (+ 309us) client_negotiation.cc:769] Sending connection context
0811 18:58:26.023870 (+ 189us) client_negotiation.cc:240] Negotiation successful
0811 18:58:26.024230 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":157,"threads_started":1}
I20260811 18:58:26.026019 26234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:26.007821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.129:53889 (local address 127.27.180.190:33249)
0811 18:58:26.008312 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:26.008331 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:26.008491 (+ 160us) server_negotiation.cc:407] Connection header received
0811 18:58:26.009972 (+ 1481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:26.009988 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:26.010389 (+ 401us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:26.010756 (+ 367us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:26.012346 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:26.013828 (+ 1482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:26.015397 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:26.016057 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:26.016216 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:26.019236 (+ 3020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:58:26.019242 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:58:26.019258 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:58:26.019293 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:58:26.021415 (+ 2122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:26.022099 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:26.022105 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:26.022111 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:26.022284 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:26.022701 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:26.022704 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:26.022706 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:26.023214 (+ 508us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:58:26.024730 (+ 1516us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:26.024880 (+ 150us) server_negotiation.cc:299] Negotiation successful
0811 18:58:26.025219 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":123,"threads_started":1}
I20260811 18:58:26.027305 26051 raft_consensus.cc:348] T ba039a5464b44ac3aa5e7e08f9acc22c P b9f8880aef354e3daf773efd8c99ad5c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d825edd376d9480f8142afe7eefab04f" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 44763 } } peers { permanent_uuid: "650f778bf77344779339041522b0028a" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 44747 } } peers { permanent_uuid: "b9f8880aef354e3daf773efd8c99ad5c" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 40553 } }
I20260811 18:58:26.027819 26051 raft_consensus.cc:729] T ba039a5464b44ac3aa5e7e08f9acc22c P b9f8880aef354e3daf773efd8c99ad5c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9f8880aef354e3daf773efd8c99ad5c, State: Initialized, Role: FOLLOWER
I20260811 18:58:26.028313 26051 consensus_queue.cc:260] T ba039a5464b44ac3aa5e7e08f9acc22c P b9f8880aef354e3daf773efd8c99ad5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d825edd376d9480f8142afe7eefab04f" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 44763 } } peers { permanent_uuid: "650f778bf77344779339041522b0028a" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 44747 } } peers { permanent_uuid: "b9f8880aef354e3daf773efd8c99ad5c" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 40553 } }
I20260811 18:58:26.028543 26229 heartbeater.cc:347] Connected to a master server at 127.27.180.190:33249
I20260811 18:58:26.028771 26229 heartbeater.cc:464] Registering TS with master...
I20260811 18:58:26.029349 26229 heartbeater.cc:511] Master 127.27.180.190:33249 requested a full tablet report, sending...
I20260811 18:58:26.029361 26051 ts_tablet_manager.cc:1428] T ba039a5464b44ac3aa5e7e08f9acc22c P b9f8880aef354e3daf773efd8c99ad5c: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260811 18:58:26.031817 25913 ts_manager.cc:194] Registered new tserver with Master: b9f8880aef354e3daf773efd8c99ad5c (127.27.180.129:40553)
I20260811 18:58:26.035271 25913 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.180.129@KRBTEST.COM'} at 127.27.180.129:53889
I20260811 18:58:26.037856 26229 heartbeater.cc:503] Master 127.27.180.190:33249 was elected leader, sending a full tablet report...
I20260811 18:58:26.041210 26240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:26.034639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.129:40553 (local address 127.0.0.1:52014)
0811 18:58:26.036327 (+ 1688us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:26.036384 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:26.036524 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:26.037178 (+ 654us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:26.037184 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:26.037211 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:58:26.037334 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:26.037341 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:26.038644 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:26.038648 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:26.039765 (+ 1117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:26.039776 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:26.040751 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:26.040754 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:26.040837 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:26.040888 (+ 51us) client_negotiation.cc:769] Sending connection context
0811 18:58:26.040971 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 18:58:26.041064 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1526,"thread_start_us":96,"threads_started":1}
I20260811 18:58:26.041260 26241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:26.034726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52014 (local address 127.27.180.129:40553)
0811 18:58:26.036090 (+ 1364us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:26.036100 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:26.036480 (+ 380us) server_negotiation.cc:407] Connection header received
0811 18:58:26.036687 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:26.036692 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:26.036989 (+ 297us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:26.037181 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:58:26.037464 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:26.038529 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:26.039881 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:26.040623 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:26.040766 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:26.040883 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:26.041011 (+ 128us) server_negotiation.cc:299] Negotiation successful
0811 18:58:26.041128 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1262,"thread_start_us":113,"threads_started":1}
I20260811 18:58:26.042181 26121 tablet_service.cc:1514] Processing DeleteTablet for tablet ba039a5464b44ac3aa5e7e08f9acc22c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:58:05 UTC) from {username='kudu', principal='kudu/127.27.180.190@KRBTEST.COM'} at 127.0.0.1:52014
I20260811 18:58:26.042987 26244 tablet_replica.cc:330] stopping tablet replica
I20260811 18:58:26.043255 26244 raft_consensus.cc:2229] T ba039a5464b44ac3aa5e7e08f9acc22c P b9f8880aef354e3daf773efd8c99ad5c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:58:26.043478 26244 raft_consensus.cc:2258] T ba039a5464b44ac3aa5e7e08f9acc22c P b9f8880aef354e3daf773efd8c99ad5c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:58:26.044386 26244 ts_tablet_manager.cc:1905] T ba039a5464b44ac3aa5e7e08f9acc22c P b9f8880aef354e3daf773efd8c99ad5c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:58:26.047859 26244 ts_tablet_manager.cc:1918] T ba039a5464b44ac3aa5e7e08f9acc22c P b9f8880aef354e3daf773efd8c99ad5c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:58:26.047940 26244 log.cc:1196] T ba039a5464b44ac3aa5e7e08f9acc22c P b9f8880aef354e3daf773efd8c99ad5c: Deleting WAL directory at /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-0/wal/wals/ba039a5464b44ac3aa5e7e08f9acc22c
I20260811 18:58:26.048163 26244 ts_tablet_manager.cc:1939] T ba039a5464b44ac3aa5e7e08f9acc22c P b9f8880aef354e3daf773efd8c99ad5c: Deleting consensus metadata
I20260811 18:58:26.049731 25889 catalog_manager.cc:4841] TS b9f8880aef354e3daf773efd8c99ad5c (127.27.180.129:40553): tablet ba039a5464b44ac3aa5e7e08f9acc22c (table test-table [id=ae201b4e25a843db854c981be171a570]) successfully deleted
W20260811 18:58:26.127955 26237 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:58:26.128196 26237 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:58:26.128248 26237 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:58:26.132550 26237 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:58:26.132609 26237 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:58:26.132683 26237 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.180.130
I20260811 18:58:26.138628 26237 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:34601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.130.keytab
--principal=kudu/127.27.180.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.180.130:44747
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.180.130
--webserver_port=34561
--tserver_master_addrs=127.27.180.190:33249
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26237
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.180.130
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:58:26.139608 26237 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:58:26.140638 26237 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:58:26.149894 26253 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:26.149906 26255 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:26.149936 26252 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:58:26.150081 26237 server_base.cc:1034] running on GCE node
I20260811 18:58:26.150470 26237 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:58:26.151067 26237 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:58:26.152215 26237 hybrid_clock.cc:648] HybridClock initialized: now 1786474706152201 us; error 25 us; skew 500 ppm
Aug 11 18:58:26 dist-test-slave-2j7r krb5kdc[24908](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474706, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:58:26.154877 26237 init.cc:377] Logged in from keytab as kudu/127.27.180.130@KRBTEST.COM (short username kudu)
I20260811 18:58:26.155740 26237 webserver.cc:460] Webserver started at http://127.27.180.130:34561/ using document root <none> and password file <none>
I20260811 18:58:26.156262 26237 fs_manager.cc:359] Metadata directory not provided
I20260811 18:58:26.156343 26237 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:58:26.160076 26237 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260811 18:58:26.162009 26268 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:26.163157 26237 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 18:58:26.163262 26237 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-1/wal
uuid: "650f778bf77344779339041522b0028a"
format_stamp: "Formatted at 2026-08-11 18:58:04 on dist-test-slave-2j7r"
server_key: "8136d360e419619e8d3e7e7ea272c48a"
server_key_iv: "14844aea4a82af0f1e6fc2d0c734fe6e"
server_key_version: "encryptionkey@0"
I20260811 18:58:26.163767 26237 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:58:26.182574 26237 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:58:26.185791 26237 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:58:26.185987 26237 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:58:26.186575 26237 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:58:26.187397 26237 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:58:26.187444 26237 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:26.187485 26237 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:58:26.187515 26237 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:26.196223 26237 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.130:44747
I20260811 18:58:26.196252 26478 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.130:44747 every 8 connection(s)
I20260811 18:58:26.197244 26237 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-1/data/info.pb
I20260811 18:58:26.200070 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 26237
I20260811 18:58:26.200579 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.180.131:44763
--local_ip_for_outbound_sockets=127.27.180.131
--tserver_master_addrs=127.27.180.190:33249
--webserver_port=40529
--webserver_interface=127.27.180.131
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.131.keytab
--principal=kudu/127.27.180.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:34601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 18:58:26 dist-test-slave-2j7r krb5kdc[24908](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474706, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.130@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM
I20260811 18:58:26.211691 26234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:26.199071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.130:35343 (local address 127.27.180.190:33249)
0811 18:58:26.199272 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:26.199278 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:26.200364 (+ 1086us) server_negotiation.cc:407] Connection header received
0811 18:58:26.201329 (+ 965us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:26.201333 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:26.201440 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:26.201515 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:26.202706 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:26.203234 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:26.204420 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:26.204722 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:26.204825 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:26.207704 (+ 2879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:58:26.207709 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:58:26.207713 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:58:26.207748 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:58:26.209448 (+ 1700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:26.209994 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:26.209997 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:26.209999 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:26.210103 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:26.210489 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:26.210492 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:26.210494 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:26.210745 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:58:26.210858 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:26.211362 (+ 504us) server_negotiation.cc:299] Negotiation successful
0811 18:58:26.211513 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260811 18:58:26.212457 26486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:26.199515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33249 (local address 127.27.180.130:35343)
0811 18:58:26.200195 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:26.200253 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:26.201124 (+ 871us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:26.201710 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:26.201717 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:26.202081 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:58:26.202551 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:26.202564 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:26.203359 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:26.203363 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:26.204280 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:26.204290 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:26.204907 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:26.204911 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:26.205073 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:26.205709 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:58:26.205727 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:58:26.207495 (+ 1768us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:58:26.209581 (+ 2086us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:58:26.209586 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:58:26.209599 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:58:26.209881 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:58:26.210231 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:58:26.210234 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:58:26.210236 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:58:26.210390 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:58:26.210844 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:58:26.210849 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:58:26.211098 (+ 249us) client_negotiation.cc:769] Sending connection context
0811 18:58:26.211279 (+ 181us) client_negotiation.cc:240] Negotiation successful
0811 18:58:26.211555 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":119,"threads_started":1}
I20260811 18:58:26.213807 26483 heartbeater.cc:347] Connected to a master server at 127.27.180.190:33249
I20260811 18:58:26.214072 26483 heartbeater.cc:464] Registering TS with master...
I20260811 18:58:26.214646 26483 heartbeater.cc:511] Master 127.27.180.190:33249 requested a full tablet report, sending...
I20260811 18:58:26.215790 25913 ts_manager.cc:194] Registered new tserver with Master: 650f778bf77344779339041522b0028a (127.27.180.130:44747)
I20260811 18:58:26.216414 25913 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.180.130@KRBTEST.COM'} at 127.27.180.130:35343
W20260811 18:58:26.316893 26489 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:58:26.317174 26489 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:58:26.317224 26489 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:58:26.321234 26489 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:58:26.321288 26489 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:58:26.321354 26489 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.180.131
I20260811 18:58:26.326514 26489 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:34601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.131.keytab
--principal=kudu/127.27.180.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.180.131:44763
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.180.131
--webserver_port=40529
--tserver_master_addrs=127.27.180.190:33249
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26489
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.180.131
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:58:26.327520 26489 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:58:26.328543 26489 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 18:58:26.338210 26489 server_base.cc:1034] running on GCE node
W20260811 18:58:26.338179 26501 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:26.338443 26503 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:26.338486 26499 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:58:26.338862 26489 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:58:26.339462 26489 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:58:26.340605 26489 hybrid_clock.cc:648] HybridClock initialized: now 1786474706340580 us; error 38 us; skew 500 ppm
Aug 11 18:58:26 dist-test-slave-2j7r krb5kdc[24908](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474706, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:58:26.343266 26489 init.cc:377] Logged in from keytab as kudu/127.27.180.131@KRBTEST.COM (short username kudu)
I20260811 18:58:26.344127 26489 webserver.cc:460] Webserver started at http://127.27.180.131:40529/ using document root <none> and password file <none>
I20260811 18:58:26.344676 26489 fs_manager.cc:359] Metadata directory not provided
I20260811 18:58:26.344758 26489 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:58:26.348593 26489 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260811 18:58:26.350607 26516 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:26.351779 26489 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 18:58:26.351890 26489 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-2/wal
uuid: "d825edd376d9480f8142afe7eefab04f"
format_stamp: "Formatted at 2026-08-11 18:58:05 on dist-test-slave-2j7r"
server_key: "0ae9bcf90361456637956151819b6750"
server_key_iv: "39694be8b7e26463ff6cbc71b959537f"
server_key_version: "encryptionkey@0"
I20260811 18:58:26.352438 26489 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:58:26.368356 26489 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:58:26.371537 26489 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:58:26.371718 26489 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:58:26.372319 26489 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:58:26.373198 26489 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:58:26.373246 26489 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:26.373301 26489 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:58:26.373325 26489 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:26.381891 26489 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.131:44763
I20260811 18:58:26.381924 26714 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.131:44763 every 8 connection(s)
I20260811 18:58:26.382875 26489 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-2/data/info.pb
I20260811 18:58:26.389199 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 26489
Aug 11 18:58:26 dist-test-slave-2j7r krb5kdc[24908](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474706, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.131@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM
I20260811 18:58:26.396371 26234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:26.384710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.131:39857 (local address 127.27.180.190:33249)
0811 18:58:26.384855 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:26.384860 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:26.385989 (+ 1129us) server_negotiation.cc:407] Connection header received
0811 18:58:26.386805 (+ 816us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:26.386807 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:26.386857 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:26.386929 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:26.388038 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:26.388560 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:26.389801 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:26.390117 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:26.390202 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:26.392953 (+ 2751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:58:26.392956 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:58:26.392958 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:58:26.392984 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:58:26.394311 (+ 1327us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:26.394876 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:26.394879 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:26.394881 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:26.395011 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:26.395317 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:26.395321 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:26.395323 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:26.395505 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:58:26.395595 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:26.396070 (+ 475us) server_negotiation.cc:299] Negotiation successful
0811 18:58:26.396205 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 18:58:26.397384 26720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:26.385181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33249 (local address 127.27.180.131:39857)
0811 18:58:26.385826 (+ 645us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:26.385868 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:26.386640 (+ 772us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:26.387083 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:26.387089 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:26.387448 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:58:26.387921 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:26.387934 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:26.388649 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:26.388652 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:26.389606 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:26.389617 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:26.390272 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:26.390276 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:26.390482 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:26.391072 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:58:26.391092 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:58:26.392790 (+ 1698us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:58:26.394452 (+ 1662us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:58:26.394460 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:58:26.394476 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:58:26.394800 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:58:26.395097 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:58:26.395098 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:58:26.395100 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:58:26.395226 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:58:26.395608 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:58:26.395615 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:58:26.395841 (+ 226us) client_negotiation.cc:769] Sending connection context
0811 18:58:26.396005 (+ 164us) client_negotiation.cc:240] Negotiation successful
0811 18:58:26.396315 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":114,"threads_started":1}
I20260811 18:58:26.398036 26721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:26.389836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40136 (local address 127.27.180.190:33249)
0811 18:58:26.390084 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:26.390089 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:26.390274 (+ 185us) server_negotiation.cc:407] Connection header received
0811 18:58:26.390387 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:26.390389 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:26.390426 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:26.390492 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:26.391383 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:26.392021 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:26.393113 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:26.393402 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:26.393509 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:26.394830 (+ 1321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:58:26.394832 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:58:26.394834 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:58:26.394874 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:58:26.396203 (+ 1329us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:26.396656 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:26.396660 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:26.396662 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:26.396745 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:26.397007 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:26.397010 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:26.397012 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:26.397249 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:58:26.397460 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:26.397720 (+ 260us) server_negotiation.cc:299] Negotiation successful
0811 18:58:26.397875 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":179,"thread_start_us":89,"threads_started":1}
I20260811 18:58:26.398693 26715 heartbeater.cc:347] Connected to a master server at 127.27.180.190:33249
I20260811 18:58:26.398933 26715 heartbeater.cc:464] Registering TS with master...
I20260811 18:58:26.399427 26715 heartbeater.cc:511] Master 127.27.180.190:33249 requested a full tablet report, sending...
I20260811 18:58:26.400431 25913 ts_manager.cc:194] Registered new tserver with Master: d825edd376d9480f8142afe7eefab04f (127.27.180.131:44763)
I20260811 18:58:26.400959 25913 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.180.131@KRBTEST.COM'} at 127.27.180.131:39857
I20260811 18:58:26.411621 28370 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 18:58:26.415427 26721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:26.412282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40150 (local address 127.27.180.190:33249)
0811 18:58:26.412416 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:26.412420 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:26.412591 (+ 171us) server_negotiation.cc:407] Connection header received
0811 18:58:26.412754 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:26.412756 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:26.412873 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:26.412955 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:58:26.413185 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:26.413695 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:26.414458 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:26.414674 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:26.414742 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:26.414981 (+ 239us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:58:26.415121 (+ 140us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0811 18:58:26.415321 (+ 200us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:40150: token expired
Metrics: {"server-negotiator.queue_time_us":50}
W20260811 18:58:26.415495 26721 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:40150: token expired
W20260811 18:58:26.415814 25791 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.27.180.190:33249)
I20260811 18:58:26.422922 26721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:26.416159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40160 (local address 127.27.180.190:33249)
0811 18:58:26.416268 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:26.416272 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:26.416285 (+ 13us) server_negotiation.cc:407] Connection header received
0811 18:58:26.416416 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:26.416419 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:26.416460 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:26.416520 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:26.417156 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:26.417634 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:26.418420 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:26.418627 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:26.418711 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:26.419971 (+ 1260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:58:26.419975 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:58:26.419977 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:58:26.420003 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:58:26.421343 (+ 1340us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:26.421744 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:26.421747 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:26.421748 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:26.421826 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:26.422081 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:26.422084 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:26.422086 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:26.422299 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:58:26.422419 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:26.422651 (+ 232us) server_negotiation.cc:299] Negotiation successful
0811 18:58:26.422765 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 18:58:26.425380 25913 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40160:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:58:26.428653 25913 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:58:26.436693 26121 tablet_service.cc:1467] Processing CreateTablet for tablet d6c21764809c449998d4df46a5da878e (DEFAULT_TABLE table=test-table [id=774bd85a9d2347369d529983e83f6d7b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:58:26.437466 26121 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6c21764809c449998d4df46a5da878e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:58:26.440682 26051 tablet_bootstrap.cc:492] T d6c21764809c449998d4df46a5da878e P b9f8880aef354e3daf773efd8c99ad5c: Bootstrap starting.
I20260811 18:58:26.440733 26732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:26.436031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.130:44747 (local address 127.0.0.1:45010)
0811 18:58:26.436302 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:26.436316 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:26.436403 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:26.437140 (+ 737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:26.437143 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:26.437164 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:58:26.437283 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:26.437290 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:26.438280 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:26.438283 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:26.439175 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:26.439186 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:26.440280 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:26.440284 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:26.440370 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:26.440399 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 18:58:26.440475 (+ 76us) client_negotiation.cc:240] Negotiation successful
0811 18:58:26.440549 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":208,"mutex_wait_us":1,"thread_start_us":120,"threads_started":1}
I20260811 18:58:26.440770 26735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:26.436226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45010 (local address 127.27.180.130:44747)
0811 18:58:26.436677 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:26.436687 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:26.436708 (+ 21us) server_negotiation.cc:407] Connection header received
0811 18:58:26.436775 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:26.436780 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:26.436985 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:26.437176 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:58:26.437379 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:26.438170 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:26.439305 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:26.440142 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:26.440271 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:26.440419 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:26.440522 (+ 103us) server_negotiation.cc:299] Negotiation successful
0811 18:58:26.440624 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":112,"threads_started":1}
I20260811 18:58:26.441118 26240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:26.435643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.131:44763 (local address 127.0.0.1:53732)
0811 18:58:26.435777 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:26.435792 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:26.435880 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:26.437219 (+ 1339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:26.437222 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:26.437242 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:58:26.437333 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:26.437339 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:26.438968 (+ 1629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:26.438971 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:26.439806 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:26.439816 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:26.440749 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:26.440753 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:26.440834 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:26.440858 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 18:58:26.440905 (+ 47us) client_negotiation.cc:240] Negotiation successful
0811 18:58:26.440973 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45,"mutex_wait_us":36}
I20260811 18:58:26.441649 26051 tablet_bootstrap.cc:654] T d6c21764809c449998d4df46a5da878e P b9f8880aef354e3daf773efd8c99ad5c: Neither blocks nor log segments found. Creating new log.
I20260811 18:58:26.442029 26736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:26.436355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53732 (local address 127.27.180.131:44763)
0811 18:58:26.436678 (+ 323us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:26.436690 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:26.436714 (+ 24us) server_negotiation.cc:407] Connection header received
0811 18:58:26.436790 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:26.436795 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:26.437014 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:26.437622 (+ 608us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:58:26.437801 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:26.438817 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:26.439924 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:26.440621 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:26.441622 (+ 1001us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:26.441754 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:26.441803 (+ 49us) server_negotiation.cc:299] Negotiation successful
0811 18:58:26.441891 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":216,"thread_start_us":108,"threads_started":1}
I20260811 18:58:26.443115 26051 tablet_bootstrap.cc:492] T d6c21764809c449998d4df46a5da878e P b9f8880aef354e3daf773efd8c99ad5c: No bootstrap required, opened a new log
I20260811 18:58:26.443216 26051 ts_tablet_manager.cc:1397] T d6c21764809c449998d4df46a5da878e P b9f8880aef354e3daf773efd8c99ad5c: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260811 18:58:26.443966 26051 raft_consensus.cc:348] T d6c21764809c449998d4df46a5da878e P b9f8880aef354e3daf773efd8c99ad5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d825edd376d9480f8142afe7eefab04f" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 44763 } } peers { permanent_uuid: "650f778bf77344779339041522b0028a" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 44747 } } peers { permanent_uuid: "b9f8880aef354e3daf773efd8c99ad5c" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 40553 } }
I20260811 18:58:26.444099 26051 raft_consensus.cc:374] T d6c21764809c449998d4df46a5da878e P b9f8880aef354e3daf773efd8c99ad5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:58:26.444136 26051 raft_consensus.cc:729] T d6c21764809c449998d4df46a5da878e P b9f8880aef354e3daf773efd8c99ad5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9f8880aef354e3daf773efd8c99ad5c, State: Initialized, Role: FOLLOWER
I20260811 18:58:26.444295 26051 consensus_queue.cc:260] T d6c21764809c449998d4df46a5da878e P b9f8880aef354e3daf773efd8c99ad5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d825edd376d9480f8142afe7eefab04f" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 44763 } } peers { permanent_uuid: "650f778bf77344779339041522b0028a" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 44747 } } peers { permanent_uuid: "b9f8880aef354e3daf773efd8c99ad5c" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 40553 } }
I20260811 18:58:26.444667 26051 ts_tablet_manager.cc:1428] T d6c21764809c449998d4df46a5da878e P b9f8880aef354e3daf773efd8c99ad5c: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260811 18:58:26.444399 26366 tablet_service.cc:1467] Processing CreateTablet for tablet d6c21764809c449998d4df46a5da878e (DEFAULT_TABLE table=test-table [id=774bd85a9d2347369d529983e83f6d7b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:58:26.444458 26601 tablet_service.cc:1467] Processing CreateTablet for tablet d6c21764809c449998d4df46a5da878e (DEFAULT_TABLE table=test-table [id=774bd85a9d2347369d529983e83f6d7b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:58:26.445534 26601 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6c21764809c449998d4df46a5da878e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:58:26.445568 26366 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6c21764809c449998d4df46a5da878e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:58:26.452651 26738 tablet_bootstrap.cc:492] T d6c21764809c449998d4df46a5da878e P 650f778bf77344779339041522b0028a: Bootstrap starting.
I20260811 18:58:26.455096 26738 tablet_bootstrap.cc:654] T d6c21764809c449998d4df46a5da878e P 650f778bf77344779339041522b0028a: Neither blocks nor log segments found. Creating new log.
I20260811 18:58:26.455960 26738 log.cc:824] T d6c21764809c449998d4df46a5da878e P 650f778bf77344779339041522b0028a: Log is configured to *not* fsync() on all Append() calls
I20260811 18:58:26.456625 26739 tablet_bootstrap.cc:492] T d6c21764809c449998d4df46a5da878e P d825edd376d9480f8142afe7eefab04f: Bootstrap starting.
I20260811 18:58:26.458022 26738 tablet_bootstrap.cc:492] T d6c21764809c449998d4df46a5da878e P 650f778bf77344779339041522b0028a: No bootstrap required, opened a new log
I20260811 18:58:26.458205 26738 ts_tablet_manager.cc:1397] T d6c21764809c449998d4df46a5da878e P 650f778bf77344779339041522b0028a: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 18:58:26.458913 26739 tablet_bootstrap.cc:654] T d6c21764809c449998d4df46a5da878e P d825edd376d9480f8142afe7eefab04f: Neither blocks nor log segments found. Creating new log.
I20260811 18:58:26.459762 26739 log.cc:824] T d6c21764809c449998d4df46a5da878e P d825edd376d9480f8142afe7eefab04f: Log is configured to *not* fsync() on all Append() calls
I20260811 18:58:26.461293 26738 raft_consensus.cc:348] T d6c21764809c449998d4df46a5da878e P 650f778bf77344779339041522b0028a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d825edd376d9480f8142afe7eefab04f" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 44763 } } peers { permanent_uuid: "650f778bf77344779339041522b0028a" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 44747 } } peers { permanent_uuid: "b9f8880aef354e3daf773efd8c99ad5c" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 40553 } }
I20260811 18:58:26.461524 26739 tablet_bootstrap.cc:492] T d6c21764809c449998d4df46a5da878e P d825edd376d9480f8142afe7eefab04f: No bootstrap required, opened a new log
I20260811 18:58:26.461524 26738 raft_consensus.cc:374] T d6c21764809c449998d4df46a5da878e P 650f778bf77344779339041522b0028a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:58:26.461589 26738 raft_consensus.cc:729] T d6c21764809c449998d4df46a5da878e P 650f778bf77344779339041522b0028a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 650f778bf77344779339041522b0028a, State: Initialized, Role: FOLLOWER
I20260811 18:58:26.461709 26739 ts_tablet_manager.cc:1397] T d6c21764809c449998d4df46a5da878e P d825edd376d9480f8142afe7eefab04f: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 18:58:26.462016 26738 consensus_queue.cc:260] T d6c21764809c449998d4df46a5da878e P 650f778bf77344779339041522b0028a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d825edd376d9480f8142afe7eefab04f" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 44763 } } peers { permanent_uuid: "650f778bf77344779339041522b0028a" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 44747 } } peers { permanent_uuid: "b9f8880aef354e3daf773efd8c99ad5c" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 40553 } }
I20260811 18:58:26.462680 26483 heartbeater.cc:503] Master 127.27.180.190:33249 was elected leader, sending a full tablet report...
I20260811 18:58:26.462971 26738 ts_tablet_manager.cc:1428] T d6c21764809c449998d4df46a5da878e P 650f778bf77344779339041522b0028a: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260811 18:58:26.465304 26739 raft_consensus.cc:348] T d6c21764809c449998d4df46a5da878e P d825edd376d9480f8142afe7eefab04f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d825edd376d9480f8142afe7eefab04f" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 44763 } } peers { permanent_uuid: "650f778bf77344779339041522b0028a" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 44747 } } peers { permanent_uuid: "b9f8880aef354e3daf773efd8c99ad5c" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 40553 } }
I20260811 18:58:26.465561 26739 raft_consensus.cc:374] T d6c21764809c449998d4df46a5da878e P d825edd376d9480f8142afe7eefab04f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:58:26.465623 26739 raft_consensus.cc:729] T d6c21764809c449998d4df46a5da878e P d825edd376d9480f8142afe7eefab04f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d825edd376d9480f8142afe7eefab04f, State: Initialized, Role: FOLLOWER
I20260811 18:58:26.466076 26739 consensus_queue.cc:260] T d6c21764809c449998d4df46a5da878e P d825edd376d9480f8142afe7eefab04f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d825edd376d9480f8142afe7eefab04f" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 44763 } } peers { permanent_uuid: "650f778bf77344779339041522b0028a" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 44747 } } peers { permanent_uuid: "b9f8880aef354e3daf773efd8c99ad5c" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 40553 } }
I20260811 18:58:26.466745 26715 heartbeater.cc:503] Master 127.27.180.190:33249 was elected leader, sending a full tablet report...
I20260811 18:58:26.467044 26739 ts_tablet_manager.cc:1428] T d6c21764809c449998d4df46a5da878e P d825edd376d9480f8142afe7eefab04f: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260811 18:58:26.495498 26746 raft_consensus.cc:482] T d6c21764809c449998d4df46a5da878e P 650f778bf77344779339041522b0028a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:58:26.495741 26746 raft_consensus.cc:504] T d6c21764809c449998d4df46a5da878e P 650f778bf77344779339041522b0028a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d825edd376d9480f8142afe7eefab04f" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 44763 } } peers { permanent_uuid: "650f778bf77344779339041522b0028a" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 44747 } } peers { permanent_uuid: "b9f8880aef354e3daf773efd8c99ad5c" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 40553 } }
I20260811 18:58:26.496989 26746 leader_election.cc:288] T d6c21764809c449998d4df46a5da878e P 650f778bf77344779339041522b0028a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d825edd376d9480f8142afe7eefab04f (127.27.180.131:44763), b9f8880aef354e3daf773efd8c99ad5c (127.27.180.129:40553)
I20260811 18:58:26.500451 26486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:26.497243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.131:44763 (local address 127.27.180.130:34535)
0811 18:58:26.497435 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:26.497452 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:26.497544 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:26.497782 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:26.497784 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:26.497802 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:58:26.497921 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:26.497926 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:26.498635 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:26.498639 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:26.499401 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:26.499410 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:26.500038 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:26.500041 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:26.500145 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:26.500164 (+ 19us) client_negotiation.cc:769] Sending connection context
0811 18:58:26.500230 (+ 66us) client_negotiation.cc:240] Negotiation successful
0811 18:58:26.500305 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73}
I20260811 18:58:26.500451 26736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:26.497324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.130:34535 (local address 127.27.180.131:44763)
0811 18:58:26.497480 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:26.497484 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:26.497499 (+ 15us) server_negotiation.cc:407] Connection header received
0811 18:58:26.497648 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:26.497651 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:26.497693 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:26.497759 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:58:26.498026 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:26.498530 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:26.499526 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:26.499929 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:26.500034 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:26.500090 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:26.500258 (+ 168us) server_negotiation.cc:299] Negotiation successful
0811 18:58:26.500327 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 18:58:26.500725 26241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:26.497384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.130:37437 (local address 127.27.180.129:40553)
0811 18:58:26.497505 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:26.497510 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:26.497864 (+ 354us) server_negotiation.cc:407] Connection header received
0811 18:58:26.497953 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:26.497956 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:26.498004 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:26.498063 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:58:26.498324 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:26.498806 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:26.499816 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:26.500213 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:26.500320 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:26.500373 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:26.500551 (+ 178us) server_negotiation.cc:299] Negotiation successful
0811 18:58:26.500623 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260811 18:58:26.500725 26752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:26.497262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.129:40553 (local address 127.27.180.130:37437)
0811 18:58:26.497750 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:26.497761 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:26.497862 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:26.498086 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:26.498089 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:26.498104 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:58:26.498217 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:26.498222 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:26.498914 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:26.498918 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:26.499691 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:26.499699 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:26.500326 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:26.500330 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:26.500434 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:26.500458 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 18:58:26.500522 (+ 64us) client_negotiation.cc:240] Negotiation successful
0811 18:58:26.500583 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":421,"mutex_wait_us":173,"thread_start_us":121,"threads_started":1}
I20260811 18:58:26.501266 26639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6c21764809c449998d4df46a5da878e" candidate_uuid: "650f778bf77344779339041522b0028a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d825edd376d9480f8142afe7eefab04f" is_pre_election: true
I20260811 18:58:26.501461 26150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6c21764809c449998d4df46a5da878e" candidate_uuid: "650f778bf77344779339041522b0028a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9f8880aef354e3daf773efd8c99ad5c" is_pre_election: true
I20260811 18:58:26.501616 26639 raft_consensus.cc:2454] T d6c21764809c449998d4df46a5da878e P d825edd376d9480f8142afe7eefab04f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 650f778bf77344779339041522b0028a in term 0.
I20260811 18:58:26.501736 26150 raft_consensus.cc:2454] T d6c21764809c449998d4df46a5da878e P b9f8880aef354e3daf773efd8c99ad5c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 650f778bf77344779339041522b0028a in term 0.
I20260811 18:58:26.502162 26274 leader_election.cc:302] T d6c21764809c449998d4df46a5da878e P 650f778bf77344779339041522b0028a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 650f778bf77344779339041522b0028a, b9f8880aef354e3daf773efd8c99ad5c, d825edd376d9480f8142afe7eefab04f; no voters:
I20260811 18:58:26.502414 26746 raft_consensus.cc:2781] T d6c21764809c449998d4df46a5da878e P 650f778bf77344779339041522b0028a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:58:26.502492 26746 raft_consensus.cc:482] T d6c21764809c449998d4df46a5da878e P 650f778bf77344779339041522b0028a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:58:26.502542 26746 raft_consensus.cc:3037] T d6c21764809c449998d4df46a5da878e P 650f778bf77344779339041522b0028a [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:58:26.503576 26746 raft_consensus.cc:504] T d6c21764809c449998d4df46a5da878e P 650f778bf77344779339041522b0028a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d825edd376d9480f8142afe7eefab04f" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 44763 } } peers { permanent_uuid: "650f778bf77344779339041522b0028a" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 44747 } } peers { permanent_uuid: "b9f8880aef354e3daf773efd8c99ad5c" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 40553 } }
I20260811 18:58:26.503944 26746 leader_election.cc:288] T d6c21764809c449998d4df46a5da878e P 650f778bf77344779339041522b0028a [CANDIDATE]: Term 1 election: Requested vote from peers d825edd376d9480f8142afe7eefab04f (127.27.180.131:44763), b9f8880aef354e3daf773efd8c99ad5c (127.27.180.129:40553)
I20260811 18:58:26.504277 26150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6c21764809c449998d4df46a5da878e" candidate_uuid: "650f778bf77344779339041522b0028a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9f8880aef354e3daf773efd8c99ad5c"
I20260811 18:58:26.504267 26639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6c21764809c449998d4df46a5da878e" candidate_uuid: "650f778bf77344779339041522b0028a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d825edd376d9480f8142afe7eefab04f"
I20260811 18:58:26.504400 26150 raft_consensus.cc:3037] T d6c21764809c449998d4df46a5da878e P b9f8880aef354e3daf773efd8c99ad5c [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:58:26.504415 26639 raft_consensus.cc:3037] T d6c21764809c449998d4df46a5da878e P d825edd376d9480f8142afe7eefab04f [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:58:26.505462 26150 raft_consensus.cc:2454] T d6c21764809c449998d4df46a5da878e P b9f8880aef354e3daf773efd8c99ad5c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 650f778bf77344779339041522b0028a in term 1.
I20260811 18:58:26.505478 26639 raft_consensus.cc:2454] T d6c21764809c449998d4df46a5da878e P d825edd376d9480f8142afe7eefab04f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 650f778bf77344779339041522b0028a in term 1.
I20260811 18:58:26.505724 26274 leader_election.cc:302] T d6c21764809c449998d4df46a5da878e P 650f778bf77344779339041522b0028a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 650f778bf77344779339041522b0028a, b9f8880aef354e3daf773efd8c99ad5c; no voters:
I20260811 18:58:26.505899 26746 raft_consensus.cc:2781] T d6c21764809c449998d4df46a5da878e P 650f778bf77344779339041522b0028a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:58:26.506095 26746 raft_consensus.cc:686] T d6c21764809c449998d4df46a5da878e P 650f778bf77344779339041522b0028a [term 1 LEADER]: Becoming Leader. State: Replica: 650f778bf77344779339041522b0028a, State: Running, Role: LEADER
I20260811 18:58:26.506426 26746 consensus_queue.cc:237] T d6c21764809c449998d4df46a5da878e P 650f778bf77344779339041522b0028a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d825edd376d9480f8142afe7eefab04f" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 44763 } } peers { permanent_uuid: "650f778bf77344779339041522b0028a" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 44747 } } peers { permanent_uuid: "b9f8880aef354e3daf773efd8c99ad5c" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 40553 } }
W20260811 18:58:26.506984 26231 tablet.cc:2369] T d6c21764809c449998d4df46a5da878e P b9f8880aef354e3daf773efd8c99ad5c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:58:26.509596 25913 catalog_manager.cc:5495] T d6c21764809c449998d4df46a5da878e P 650f778bf77344779339041522b0028a reported cstate change: term changed from 0 to 1, leader changed from <none> to 650f778bf77344779339041522b0028a (127.27.180.130). New cstate: current_term: 1 leader_uuid: "650f778bf77344779339041522b0028a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d825edd376d9480f8142afe7eefab04f" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 44763 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "650f778bf77344779339041522b0028a" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 44747 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b9f8880aef354e3daf773efd8c99ad5c" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 40553 } health_report { overall_health: UNKNOWN } } }
I20260811 18:58:26.531903 26735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:26.528386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45026 (local address 127.27.180.130:44747)
0811 18:58:26.528506 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:26.528511 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:26.528525 (+ 14us) server_negotiation.cc:407] Connection header received
0811 18:58:26.528686 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:26.528689 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:26.528746 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:26.528835 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:58:26.529068 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:26.529584 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:26.530450 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:26.530671 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:26.530757 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:26.531062 (+ 305us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:58:26.531523 (+ 461us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:58:26.531629 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:26.531689 (+ 60us) server_negotiation.cc:299] Negotiation successful
0811 18:58:26.531745 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 18:58:26.537631 26639 raft_consensus.cc:1261] T d6c21764809c449998d4df46a5da878e P d825edd376d9480f8142afe7eefab04f [term 1 FOLLOWER]: Refusing update from remote peer 650f778bf77344779339041522b0028a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:58:26.537627 26150 raft_consensus.cc:1261] T d6c21764809c449998d4df46a5da878e P b9f8880aef354e3daf773efd8c99ad5c [term 1 FOLLOWER]: Refusing update from remote peer 650f778bf77344779339041522b0028a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:58:26.538316 26754 consensus_queue.cc:1035] T d6c21764809c449998d4df46a5da878e P 650f778bf77344779339041522b0028a [LEADER]: Connected to new peer: Peer: permanent_uuid: "b9f8880aef354e3daf773efd8c99ad5c" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 40553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:58:26.538502 26746 consensus_queue.cc:1035] T d6c21764809c449998d4df46a5da878e P 650f778bf77344779339041522b0028a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d825edd376d9480f8142afe7eefab04f" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 44763 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:58:26.543597 26757 mvcc.cc:205] Tried to move back new op lower bound from 7317400397971263488 to 7317400397853040640. Current Snapshot: MvccSnapshot[applied={T|T < 7317400397971263488}]
I20260811 18:58:26.544113 26758 mvcc.cc:205] Tried to move back new op lower bound from 7317400397971263488 to 7317400397853040640. Current Snapshot: MvccSnapshot[applied={T|T < 7317400397971263488}]
I20260811 18:58:26.553715 25913 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40160:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:58:26.553920 25913 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40160:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:58:26.556329 25913 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b3054cf295fb43d6bb0787856013df0e: Sending DeleteTablet for 3 replicas of tablet d6c21764809c449998d4df46a5da878e
I20260811 18:58:26.556993 26121 tablet_service.cc:1514] Processing DeleteTablet for tablet d6c21764809c449998d4df46a5da878e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:58:26 UTC) from {username='kudu', principal='kudu/127.27.180.190@KRBTEST.COM'} at 127.0.0.1:52014
I20260811 18:58:26.557039 26601 tablet_service.cc:1514] Processing DeleteTablet for tablet d6c21764809c449998d4df46a5da878e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:58:26 UTC) from {username='kudu', principal='kudu/127.27.180.190@KRBTEST.COM'} at 127.0.0.1:53732
I20260811 18:58:26.557053 26366 tablet_service.cc:1514] Processing DeleteTablet for tablet d6c21764809c449998d4df46a5da878e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:58:26 UTC) from {username='kudu', principal='kudu/127.27.180.190@KRBTEST.COM'} at 127.0.0.1:45010
I20260811 18:58:26.557508 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 25978
I20260811 18:58:26.557557 26772 tablet_replica.cc:330] T d6c21764809c449998d4df46a5da878e P b9f8880aef354e3daf773efd8c99ad5c: stopping tablet replica
I20260811 18:58:26.572397 26775 tablet_replica.cc:330] T d6c21764809c449998d4df46a5da878e P 650f778bf77344779339041522b0028a: stopping tablet replica
I20260811 18:58:26.572803 26773 tablet_replica.cc:330] T d6c21764809c449998d4df46a5da878e P d825edd376d9480f8142afe7eefab04f: stopping tablet replica
I20260811 18:58:26.573269 26773 raft_consensus.cc:2229] T d6c21764809c449998d4df46a5da878e P d825edd376d9480f8142afe7eefab04f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:58:26.573585 26775 raft_consensus.cc:2229] T d6c21764809c449998d4df46a5da878e P 650f778bf77344779339041522b0028a [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:58:26.574023 26775 raft_consensus.cc:2258] T d6c21764809c449998d4df46a5da878e P 650f778bf77344779339041522b0028a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:58:26.574398 26773 raft_consensus.cc:2258] T d6c21764809c449998d4df46a5da878e P d825edd376d9480f8142afe7eefab04f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:58:26.576448 26775 ts_tablet_manager.cc:1905] T d6c21764809c449998d4df46a5da878e P 650f778bf77344779339041522b0028a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:58:26.576448 26773 ts_tablet_manager.cc:1905] T d6c21764809c449998d4df46a5da878e P d825edd376d9480f8142afe7eefab04f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:58:26.579290 26773 ts_tablet_manager.cc:1918] T d6c21764809c449998d4df46a5da878e P d825edd376d9480f8142afe7eefab04f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:58:26.579290 26775 ts_tablet_manager.cc:1918] T d6c21764809c449998d4df46a5da878e P 650f778bf77344779339041522b0028a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:58:26.579416 26773 log.cc:1196] T d6c21764809c449998d4df46a5da878e P d825edd376d9480f8142afe7eefab04f: Deleting WAL directory at /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-2/wal/wals/d6c21764809c449998d4df46a5da878e
I20260811 18:58:26.579545 26775 log.cc:1196] T d6c21764809c449998d4df46a5da878e P 650f778bf77344779339041522b0028a: Deleting WAL directory at /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-1/wal/wals/d6c21764809c449998d4df46a5da878e
I20260811 18:58:26.579780 26773 ts_tablet_manager.cc:1939] T d6c21764809c449998d4df46a5da878e P d825edd376d9480f8142afe7eefab04f: Deleting consensus metadata
I20260811 18:58:26.580029 26775 ts_tablet_manager.cc:1939] T d6c21764809c449998d4df46a5da878e P 650f778bf77344779339041522b0028a: Deleting consensus metadata
I20260811 18:58:26.581229 25887 catalog_manager.cc:4841] TS d825edd376d9480f8142afe7eefab04f (127.27.180.131:44763): tablet d6c21764809c449998d4df46a5da878e (table test-table [id=774bd85a9d2347369d529983e83f6d7b]) successfully deleted
I20260811 18:58:26.581246 25891 catalog_manager.cc:4841] TS 650f778bf77344779339041522b0028a (127.27.180.130:44747): tablet d6c21764809c449998d4df46a5da878e (table test-table [id=774bd85a9d2347369d529983e83f6d7b]) successfully deleted
W20260811 18:58:26.582307 25889 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.27.180.129:40553 (error 108)
I20260811 18:58:26.582960 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 26237
I20260811 18:58:26.591777 26240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:26.583115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.129:40553 (local address 127.0.0.1:52020)
0811 18:58:26.591546 (+ 8431us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:26.591665 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.129:40553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":8306}
W20260811 18:58:26.592093 25889 catalog_manager.cc:4584] TS b9f8880aef354e3daf773efd8c99ad5c (127.27.180.129:40553): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d6c21764809c449998d4df46a5da878e: Network error: Client connection negotiation failed: client connection to 127.27.180.129:40553: connect: Connection refused (error 111)
I20260811 18:58:26.598788 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 26489
I20260811 18:58:26.614560 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 25860
I20260811 18:58:46.631867 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.190:33249
--webserver_interface=127.27.180.190
--webserver_port=33175
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab
--principal=kudu/127.27.180.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:34601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.180.190:33249
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:58:46.743937 26778 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:58:46.744189 26778 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:58:46.744234 26778 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:58:46.748189 26778 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 18:58:46.748243 26778 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 18:58:46.748258 26778 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 18:58:46.748270 26778 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:58:46.748286 26778 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:58:46.748302 26778 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:58:46.748319 26778 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:58:46.748338 26778 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:58:46.753458 26778 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:34601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.27.180.190:33249
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab
--principal=kudu/127.27.180.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.190:33249
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.180.190
--webserver_port=33175
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26778
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:58:46.754382 26778 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:58:46.755321 26778 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 18:58:46.763722 26778 server_base.cc:1034] running on GCE node
W20260811 18:58:46.763790 26788 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:46.763909 26789 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:46.763739 26796 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:58:46.764552 26778 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:58:46.765625 26778 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:58:46.766794 26778 hybrid_clock.cc:648] HybridClock initialized: now 1786474726766777 us; error 29 us; skew 500 ppm
Aug 11 18:58:46 dist-test-slave-2j7r krb5kdc[24908](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474726, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:58:46.769461 26778 init.cc:377] Logged in from keytab as kudu/127.27.180.190@KRBTEST.COM (short username kudu)
I20260811 18:58:46.770287 26778 webserver.cc:460] Webserver started at http://127.27.180.190:33175/ using document root <none> and password file <none>
I20260811 18:58:46.770807 26778 fs_manager.cc:359] Metadata directory not provided
I20260811 18:58:46.770888 26778 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:58:46.774523 26778 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260811 18:58:46.776527 26809 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:46.777670 26778 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 18:58:46.777789 26778 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/master-0/wal
uuid: "b3054cf295fb43d6bb0787856013df0e"
format_stamp: "Formatted at 2026-08-11 18:58:04 on dist-test-slave-2j7r"
server_key: "7cb1e0c8cf224b542b7cef8b2d22b75d"
server_key_iv: "4b3cfb7012c4f74187c871d278b3bb3b"
server_key_version: "encryptionkey@0"
I20260811 18:58:46.778309 26778 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:58:46.801698 26778 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:58:46.804358 26778 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:58:46.804533 26778 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:58:46.811510 26778 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.190:33249
I20260811 18:58:46.811780 26898 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.190:33249 every 8 connection(s)
I20260811 18:58:46.812525 26778 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
I20260811 18:58:46.816651 26900 sys_catalog.cc:263] Verifying existing consensus state
I20260811 18:58:46.818288 26900 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b3054cf295fb43d6bb0787856013df0e: Bootstrap starting.
I20260811 18:58:46.819293 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 26778
I20260811 18:58:46.819712 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.180.129:40553
--local_ip_for_outbound_sockets=127.27.180.129
--tserver_master_addrs=127.27.180.190:33249
--webserver_port=42531
--webserver_interface=127.27.180.129
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.129.keytab
--principal=kudu/127.27.180.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:34601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 18:58:46.837258 26900 log.cc:824] T 00000000000000000000000000000000 P b3054cf295fb43d6bb0787856013df0e: Log is configured to *not* fsync() on all Append() calls
I20260811 18:58:46.848527 26900 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b3054cf295fb43d6bb0787856013df0e: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 18:58:46.848896 26900 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b3054cf295fb43d6bb0787856013df0e: Bootstrap complete.
I20260811 18:58:46.852931 26900 raft_consensus.cc:348] T 00000000000000000000000000000000 P b3054cf295fb43d6bb0787856013df0e [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3054cf295fb43d6bb0787856013df0e" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 33249 } }
I20260811 18:58:46.853401 26900 raft_consensus.cc:729] T 00000000000000000000000000000000 P b3054cf295fb43d6bb0787856013df0e [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3054cf295fb43d6bb0787856013df0e, State: Initialized, Role: FOLLOWER
I20260811 18:58:46.853881 26900 consensus_queue.cc:260] T 00000000000000000000000000000000 P b3054cf295fb43d6bb0787856013df0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3054cf295fb43d6bb0787856013df0e" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 33249 } }
I20260811 18:58:46.854010 26900 raft_consensus.cc:388] T 00000000000000000000000000000000 P b3054cf295fb43d6bb0787856013df0e [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:58:46.854081 26900 raft_consensus.cc:482] T 00000000000000000000000000000000 P b3054cf295fb43d6bb0787856013df0e [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:58:46.854178 26900 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b3054cf295fb43d6bb0787856013df0e [term 2 FOLLOWER]: Advancing to term 3
I20260811 18:58:46.856168 26900 raft_consensus.cc:504] T 00000000000000000000000000000000 P b3054cf295fb43d6bb0787856013df0e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3054cf295fb43d6bb0787856013df0e" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 33249 } }
I20260811 18:58:46.856487 26900 leader_election.cc:302] T 00000000000000000000000000000000 P b3054cf295fb43d6bb0787856013df0e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b3054cf295fb43d6bb0787856013df0e; no voters:
I20260811 18:58:46.856794 26900 leader_election.cc:288] T 00000000000000000000000000000000 P b3054cf295fb43d6bb0787856013df0e [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 18:58:46.856904 26908 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b3054cf295fb43d6bb0787856013df0e [term 3 FOLLOWER]: Leader election won for term 3
I20260811 18:58:46.857147 26908 raft_consensus.cc:686] T 00000000000000000000000000000000 P b3054cf295fb43d6bb0787856013df0e [term 3 LEADER]: Becoming Leader. State: Replica: b3054cf295fb43d6bb0787856013df0e, State: Running, Role: LEADER
I20260811 18:58:46.857522 26908 consensus_queue.cc:237] T 00000000000000000000000000000000 P b3054cf295fb43d6bb0787856013df0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3054cf295fb43d6bb0787856013df0e" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 33249 } }
I20260811 18:58:46.858103 26900 sys_catalog.cc:564] T 00000000000000000000000000000000 P b3054cf295fb43d6bb0787856013df0e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:58:46.859050 26910 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3054cf295fb43d6bb0787856013df0e [sys.catalog]: SysCatalogTable state changed. Reason: New leader b3054cf295fb43d6bb0787856013df0e. Latest consensus state: current_term: 3 leader_uuid: "b3054cf295fb43d6bb0787856013df0e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3054cf295fb43d6bb0787856013df0e" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 33249 } } }
I20260811 18:58:46.859174 26910 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3054cf295fb43d6bb0787856013df0e [sys.catalog]: This master's current role is: LEADER
I20260811 18:58:46.859545 26921 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:58:46.859524 26909 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3054cf295fb43d6bb0787856013df0e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "b3054cf295fb43d6bb0787856013df0e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3054cf295fb43d6bb0787856013df0e" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 33249 } } }
I20260811 18:58:46.859671 26909 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3054cf295fb43d6bb0787856013df0e [sys.catalog]: This master's current role is: LEADER
I20260811 18:58:46.864779 26921 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:58:46.865623 26921 catalog_manager.cc:1259] Loaded cluster ID: a07c061746b84cfda0f8ef5a97059df7
I20260811 18:58:46.868018 26921 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:58:46.871373 26921 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:58:46.872206 26921 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b3054cf295fb43d6bb0787856013df0e: Loaded TSK: 0
I20260811 18:58:46.872351 26921 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b3054cf295fb43d6bb0787856013df0e: Loaded TSK: 1
F20260811 18:58:46.873167 26921 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f97fdec39ad google::LogMessage::Fail() at ??:0
@ 0x7f97fdec688a google::LogMessage::SendToLog() at ??:0
@ 0x7f97fdec34de google::LogMessage::Flush() at ??:0
@ 0x7f97fdec4a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f9800b8718d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f9800b7ff0d kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f98088287bb kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f980880299c kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f9808803178 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlvE3_clEv at ??:0
@ 0x7f9808844df8 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f980886382b std::function<>::operator()() at ??:0
@ 0x7f9808802b4f _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f9808803f3e kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f9808800579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f9808844640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f98001153d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f9800115c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f98001173f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f9800106512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f980174a6db start_thread at ??:0
@ 0x7f97fc9c161f clone at ??:0
W20260811 18:58:46.959583 26904 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:58:46.959928 26904 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:58:46.960001 26904 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:58:46.966046 26904 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:58:46.966117 26904 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:58:46.966207 26904 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.180.129
I20260811 18:58:46.973834 26904 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:34601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.129.keytab
--principal=kudu/127.27.180.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.180.129:40553
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.180.129
--webserver_port=42531
--tserver_master_addrs=127.27.180.190:33249
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26904
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.180.129
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:58:46.975224 26904 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:58:46.976603 26904 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:58:46.987818 26979 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:46.987927 26982 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:58:46.988054 26904 server_base.cc:1034] running on GCE node
W20260811 18:58:46.988155 26978 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:58:46.988622 26904 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:58:46.989380 26904 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:58:46.990571 26904 hybrid_clock.cc:648] HybridClock initialized: now 1786474726990547 us; error 33 us; skew 500 ppm
Aug 11 18:58:46 dist-test-slave-2j7r krb5kdc[24908](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474726, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:58:46.993782 26904 init.cc:377] Logged in from keytab as kudu/127.27.180.129@KRBTEST.COM (short username kudu)
I20260811 18:58:46.994599 26904 webserver.cc:460] Webserver started at http://127.27.180.129:42531/ using document root <none> and password file <none>
I20260811 18:58:46.995142 26904 fs_manager.cc:359] Metadata directory not provided
I20260811 18:58:46.995201 26904 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:58:46.999150 26904 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 18:58:47.001391 27002 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:47.002717 26904 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 18:58:47.002844 26904 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-0/wal
uuid: "b9f8880aef354e3daf773efd8c99ad5c"
format_stamp: "Formatted at 2026-08-11 18:58:04 on dist-test-slave-2j7r"
server_key: "9b6da8953611d63f2da567b5635eb177"
server_key_iv: "3eeb0fea774b51eedc383e0ec2a4fb46"
server_key_version: "encryptionkey@0"
I20260811 18:58:47.003372 26904 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:58:47.017318 26904 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:58:47.020474 26904 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:58:47.020649 26904 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:58:47.021292 26904 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:58:47.022387 27013 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 18:58:47.026378 26904 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 18:58:47.026436 26904 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.000s sys 0.000s
I20260811 18:58:47.026482 26904 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 18:58:47.028198 26904 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 18:58:47.028252 26904 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260811 18:58:47.028534 27013 tablet_bootstrap.cc:492] T d6c21764809c449998d4df46a5da878e P b9f8880aef354e3daf773efd8c99ad5c: Bootstrap starting.
I20260811 18:58:47.038415 26904 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.129:40553
I20260811 18:58:47.038650 27213 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.129:40553 every 8 connection(s)
I20260811 18:58:47.039639 26904 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-0/data/info.pb
I20260811 18:58:47.048002 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 26904
I20260811 18:58:47.048523 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.180.130:44747
--local_ip_for_outbound_sockets=127.27.180.130
--tserver_master_addrs=127.27.180.190:33249
--webserver_port=34561
--webserver_interface=127.27.180.130
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.130.keytab
--principal=kudu/127.27.180.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:34601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 18:58:47.049201 27013 log.cc:824] T d6c21764809c449998d4df46a5da878e P b9f8880aef354e3daf773efd8c99ad5c: Log is configured to *not* fsync() on all Append() calls
I20260811 18:58:47.057597 27013 tablet_bootstrap.cc:492] T d6c21764809c449998d4df46a5da878e P b9f8880aef354e3daf773efd8c99ad5c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 18:58:47.058043 27013 tablet_bootstrap.cc:492] T d6c21764809c449998d4df46a5da878e P b9f8880aef354e3daf773efd8c99ad5c: Bootstrap complete.
I20260811 18:58:47.059569 27013 ts_tablet_manager.cc:1397] T d6c21764809c449998d4df46a5da878e P b9f8880aef354e3daf773efd8c99ad5c: Time spent bootstrapping tablet: real 0.031s user 0.024s sys 0.004s
I20260811 18:58:47.062979 27013 raft_consensus.cc:348] T d6c21764809c449998d4df46a5da878e P b9f8880aef354e3daf773efd8c99ad5c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d825edd376d9480f8142afe7eefab04f" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 44763 } } peers { permanent_uuid: "650f778bf77344779339041522b0028a" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 44747 } } peers { permanent_uuid: "b9f8880aef354e3daf773efd8c99ad5c" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 40553 } }
I20260811 18:58:47.063534 27013 raft_consensus.cc:729] T d6c21764809c449998d4df46a5da878e P b9f8880aef354e3daf773efd8c99ad5c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9f8880aef354e3daf773efd8c99ad5c, State: Initialized, Role: FOLLOWER
I20260811 18:58:47.064095 27013 consensus_queue.cc:260] T d6c21764809c449998d4df46a5da878e P b9f8880aef354e3daf773efd8c99ad5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d825edd376d9480f8142afe7eefab04f" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 44763 } } peers { permanent_uuid: "650f778bf77344779339041522b0028a" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 44747 } } peers { permanent_uuid: "b9f8880aef354e3daf773efd8c99ad5c" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 40553 } }
I20260811 18:58:47.065171 27013 ts_tablet_manager.cc:1428] T d6c21764809c449998d4df46a5da878e P b9f8880aef354e3daf773efd8c99ad5c: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
W20260811 18:58:47.112347 27221 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:58:47.042188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33249 (local address 127.27.180.129:36071)
0811 18:58:47.042798 (+ 610us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:47.042869 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:47.043962 (+ 1093us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:47.111561 (+ 67599us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:33249: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":171,"threads_started":1}
W20260811 18:58:47.112895 27214 heartbeater.cc:644] Failed to heartbeat to 127.27.180.190:33249 (0 consecutive failures): Network error: Failed to ping master at 127.27.180.190:33249: Client connection negotiation failed: client connection to 127.27.180.190:33249: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 18:58:47.113718 27221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:47.113464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33249 (local address 127.27.180.129:46397)
0811 18:58:47.113595 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:47.113660 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:33249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260811 18:58:47.114332 27221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:47.114136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33249 (local address 127.27.180.129:40207)
0811 18:58:47.114238 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:47.114283 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:33249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260811 18:58:47.114487 27214 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 18:58:47.178218 27225 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:58:47.178489 27225 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:58:47.178552 27225 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:58:47.182613 27225 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:58:47.182669 27225 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:58:47.182770 27225 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.180.130
I20260811 18:58:47.188185 27225 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:34601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.130.keytab
--principal=kudu/127.27.180.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.180.130:44747
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.180.130
--webserver_port=34561
--tserver_master_addrs=127.27.180.190:33249
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27225
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.180.130
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:58:47.189230 27225 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:58:47.190197 27225 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:58:47.199558 27235 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:58:47.199869 27225 server_base.cc:1034] running on GCE node
W20260811 18:58:47.200016 27237 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:47.199636 27242 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:58:47.200583 27225 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:58:47.201366 27225 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:58:47.202556 27225 hybrid_clock.cc:648] HybridClock initialized: now 1786474727202524 us; error 47 us; skew 500 ppm
Aug 11 18:58:47 dist-test-slave-2j7r krb5kdc[24908](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474727, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:58:47.205366 27225 init.cc:377] Logged in from keytab as kudu/127.27.180.130@KRBTEST.COM (short username kudu)
I20260811 18:58:47.206341 27225 webserver.cc:460] Webserver started at http://127.27.180.130:34561/ using document root <none> and password file <none>
I20260811 18:58:47.206907 27225 fs_manager.cc:359] Metadata directory not provided
I20260811 18:58:47.206974 27225 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:58:47.211015 27225 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260811 18:58:47.213210 27251 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:47.214551 27225 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 18:58:47.214676 27225 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-1/wal
uuid: "650f778bf77344779339041522b0028a"
format_stamp: "Formatted at 2026-08-11 18:58:04 on dist-test-slave-2j7r"
server_key: "8136d360e419619e8d3e7e7ea272c48a"
server_key_iv: "14844aea4a82af0f1e6fc2d0c734fe6e"
server_key_version: "encryptionkey@0"
I20260811 18:58:47.215237 27225 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:58:47.236127 27225 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:58:47.238903 27225 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:58:47.239084 27225 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:58:47.239693 27225 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:58:47.240535 27225 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:58:47.240581 27225 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:47.240638 27225 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:58:47.240662 27225 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:47.249579 27225 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.130:44747
I20260811 18:58:47.249606 27463 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.130:44747 every 8 connection(s)
I20260811 18:58:47.250564 27225 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-1/data/info.pb
I20260811 18:58:47.254149 27467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:47.252583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33249 (local address 127.27.180.130:49489)
0811 18:58:47.253359 (+ 776us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:47.253464 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:33249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":583,"thread_start_us":118,"threads_started":1}
W20260811 18:58:47.254505 27464 heartbeater.cc:644] Failed to heartbeat to 127.27.180.190:33249 (0 consecutive failures): Network error: Failed to ping master at 127.27.180.190:33249: Client connection negotiation failed: client connection to 127.27.180.190:33249: connect: Connection refused (error 111)
I20260811 18:58:47.254949 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 27225
I20260811 18:58:47.255133 27467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:47.254896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33249 (local address 127.27.180.130:36121)
0811 18:58:47.255018 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:47.255074 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:33249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260811 18:58:47.255434 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.180.131:44763
--local_ip_for_outbound_sockets=127.27.180.131
--tserver_master_addrs=127.27.180.190:33249
--webserver_port=40529
--webserver_interface=127.27.180.131
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.131.keytab
--principal=kudu/127.27.180.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:34601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 18:58:47.256062 27467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:47.255843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33249 (local address 127.27.180.130:58247)
0811 18:58:47.255948 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:47.255997 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:33249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260811 18:58:47.256219 27464 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 18:58:47.366605 27468 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:58:47.366870 27468 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:58:47.366912 27468 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:58:47.370841 27468 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:58:47.370903 27468 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:58:47.370965 27468 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.180.131
I20260811 18:58:47.376173 27468 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:34601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.131.keytab
--principal=kudu/127.27.180.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.180.131:44763
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.180.131
--webserver_port=40529
--tserver_master_addrs=127.27.180.190:33249
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27468
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.180.131
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:58:47.377231 27468 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:58:47.378347 27468 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:58:47.388091 27478 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:47.388106 27481 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:58:47.388132 27485 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:58:47.388305 27468 server_base.cc:1034] running on GCE node
I20260811 18:58:47.388777 27468 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:58:47.389415 27468 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:58:47.390583 27468 hybrid_clock.cc:648] HybridClock initialized: now 1786474727390562 us; error 31 us; skew 500 ppm
Aug 11 18:58:47 dist-test-slave-2j7r krb5kdc[24908](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474727, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:58:47.393376 27468 init.cc:377] Logged in from keytab as kudu/127.27.180.131@KRBTEST.COM (short username kudu)
I20260811 18:58:47.394219 27468 webserver.cc:460] Webserver started at http://127.27.180.131:40529/ using document root <none> and password file <none>
I20260811 18:58:47.394773 27468 fs_manager.cc:359] Metadata directory not provided
I20260811 18:58:47.394840 27468 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:58:47.398571 27468 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.001s
I20260811 18:58:47.400544 27501 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:47.401752 27468 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 18:58:47.401881 27468 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-2/wal
uuid: "d825edd376d9480f8142afe7eefab04f"
format_stamp: "Formatted at 2026-08-11 18:58:05 on dist-test-slave-2j7r"
server_key: "0ae9bcf90361456637956151819b6750"
server_key_iv: "39694be8b7e26463ff6cbc71b959537f"
server_key_version: "encryptionkey@0"
I20260811 18:58:47.402408 27468 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:58:47.413990 27468 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:58:47.416832 27468 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:58:47.416978 27468 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:58:47.417555 27468 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:58:47.418356 27468 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:58:47.418402 27468 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:47.418440 27468 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:58:47.418460 27468 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:58:47.427166 27468 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.131:44763
I20260811 18:58:47.427189 27703 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.131:44763 every 8 connection(s)
I20260811 18:58:47.428128 27468 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786474496157106-28370-0/minicluster-data/ts-2/data/info.pb
I20260811 18:58:47.431793 27709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:47.430335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33249 (local address 127.27.180.131:55553)
0811 18:58:47.431005 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:47.431109 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:33249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":130,"threads_started":1}
W20260811 18:58:47.432279 27704 heartbeater.cc:644] Failed to heartbeat to 127.27.180.190:33249 (0 consecutive failures): Network error: Failed to ping master at 127.27.180.190:33249: Client connection negotiation failed: client connection to 127.27.180.190:33249: connect: Connection refused (error 111)
I20260811 18:58:47.432503 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 27468
I20260811 18:58:47.432937 27709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:47.432685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33249 (local address 127.27.180.131:55283)
0811 18:58:47.432804 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:47.432878 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:33249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260811 18:58:47.433535 27709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:47.433333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33249 (local address 127.27.180.131:55199)
0811 18:58:47.433440 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:47.433486 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:33249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 18:58:47.433701 27704 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 18:58:48.115631 27711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:48.115093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33249 (local address 127.27.180.129:50791)
0811 18:58:48.115436 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:48.115502 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:33249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":78,"threads_started":1}
I20260811 18:58:48.257140 27713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:48.256702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33249 (local address 127.27.180.130:44081)
0811 18:58:48.256957 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:48.257008 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:33249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":71,"threads_started":1}
I20260811 18:58:48.434602 27721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:48.434188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33249 (local address 127.27.180.131:33185)
0811 18:58:48.434452 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:48.434505 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:33249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":95,"threads_started":1}
I20260811 18:58:48.545964 27722 raft_consensus.cc:482] T d6c21764809c449998d4df46a5da878e P b9f8880aef354e3daf773efd8c99ad5c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:58:48.546159 27722 raft_consensus.cc:504] T d6c21764809c449998d4df46a5da878e P b9f8880aef354e3daf773efd8c99ad5c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d825edd376d9480f8142afe7eefab04f" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 44763 } } peers { permanent_uuid: "650f778bf77344779339041522b0028a" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 44747 } } peers { permanent_uuid: "b9f8880aef354e3daf773efd8c99ad5c" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 40553 } }
I20260811 18:58:48.547205 27722 leader_election.cc:288] T d6c21764809c449998d4df46a5da878e P b9f8880aef354e3daf773efd8c99ad5c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d825edd376d9480f8142afe7eefab04f (127.27.180.131:44763), 650f778bf77344779339041522b0028a (127.27.180.130:44747)
Aug 11 18:58:48 dist-test-slave-2j7r krb5kdc[24908](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474726, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.129@KRBTEST.COM for kudu/127.27.180.130@KRBTEST.COM
Aug 11 18:58:48 dist-test-slave-2j7r krb5kdc[24908](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474726, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.129@KRBTEST.COM for kudu/127.27.180.131@KRBTEST.COM
I20260811 18:58:48.560467 27723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:48.547472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.130:44747 (local address 127.27.180.129:43335)
0811 18:58:48.547854 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:48.547868 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:48.547972 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:48.549460 (+ 1488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:48.549466 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:48.550014 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:58:48.550492 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:48.550506 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:48.551738 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:48.551740 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:48.552689 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:48.552699 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:48.553408 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:48.553412 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:48.553659 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:48.554225 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:58:48.554280 (+ 55us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:58:48.555724 (+ 1444us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:58:48.557946 (+ 2222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:58:48.557951 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:58:48.557965 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:58:48.558298 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:58:48.558654 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:58:48.558656 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:58:48.558657 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:58:48.558802 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:58:48.559492 (+ 690us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:58:48.559498 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:58:48.559734 (+ 236us) client_negotiation.cc:769] Sending connection context
0811 18:58:48.559928 (+ 194us) client_negotiation.cc:240] Negotiation successful
0811 18:58:48.560222 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"mutex_wait_us":213,"spinlock_wait_cycles":57472,"thread_start_us":80,"threads_started":1}
I20260811 18:58:48.560606 27729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:48.547621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.129:43335 (local address 127.27.180.130:44747)
0811 18:58:48.548041 (+ 420us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:48.548065 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:48.548098 (+ 33us) server_negotiation.cc:407] Connection header received
0811 18:58:48.548746 (+ 648us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:48.548751 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:48.549074 (+ 323us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:48.549389 (+ 315us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:48.550654 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:48.551599 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:48.552802 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:48.553295 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:48.553440 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:48.555995 (+ 2555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:58:48.556002 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:58:48.556018 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:58:48.556051 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:58:48.557832 (+ 1781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:48.558407 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:48.558412 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:48.558416 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:48.558556 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:48.558894 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:48.558897 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:48.558898 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:48.559368 (+ 470us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:58:48.559460 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:48.560107 (+ 647us) server_negotiation.cc:299] Negotiation successful
0811 18:58:48.560435 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":60,"threads_started":1}
I20260811 18:58:48.560607 27730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:48.547615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.129:60123 (local address 127.27.180.131:44763)
0811 18:58:48.547991 (+ 376us) server_negotiation.cc:206] Beginning negotiation
0811 18:58:48.548015 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0811 18:58:48.548051 (+ 36us) server_negotiation.cc:407] Connection header received
0811 18:58:48.548740 (+ 689us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:58:48.548747 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:58:48.549073 (+ 326us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:58:48.549388 (+ 315us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:58:48.550657 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:48.551611 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:48.552811 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:58:48.553295 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:58:48.553447 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:48.556372 (+ 2925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:58:48.556378 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:58:48.556392 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:58:48.556641 (+ 249us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:58:48.558403 (+ 1762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:48.558837 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:48.558843 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:48.558848 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:48.558968 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:58:48.559253 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:58:48.559255 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:58:48.559256 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:58:48.559676 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:58:48.559815 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0811 18:58:48.560180 (+ 365us) server_negotiation.cc:299] Negotiation successful
0811 18:58:48.560446 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273,"thread_start_us":124,"threads_started":1}
I20260811 18:58:48.560503 27711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:48.547401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.131:44763 (local address 127.27.180.129:60123)
0811 18:58:48.547638 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:48.547652 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 18:58:48.547759 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:58:48.549461 (+ 1702us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:58:48.549466 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:58:48.550014 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:58:48.550492 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:48.550506 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:48.551740 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:48.551743 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:48.552686 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:58:48.552695 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:58:48.553404 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:58:48.553408 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:58:48.553657 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:58:48.554280 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:58:48.554292 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:58:48.556142 (+ 1850us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:58:48.558503 (+ 2361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:58:48.558506 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:58:48.558508 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:58:48.558736 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:58:48.559080 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:58:48.559083 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:58:48.559085 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:58:48.559171 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:58:48.559830 (+ 659us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:58:48.559833 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:58:48.559931 (+ 98us) client_negotiation.cc:769] Sending connection context
0811 18:58:48.560045 (+ 114us) client_negotiation.cc:240] Negotiation successful
0811 18:58:48.560308 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"mutex_wait_us":261,"spinlock_wait_cycles":5504}
W20260811 18:58:48.563259 27004 leader_election.cc:341] T d6c21764809c449998d4df46a5da878e P b9f8880aef354e3daf773efd8c99ad5c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d825edd376d9480f8142afe7eefab04f (127.27.180.131:44763): Not found: Tablet not found: d6c21764809c449998d4df46a5da878e
W20260811 18:58:48.563450 27006 leader_election.cc:341] T d6c21764809c449998d4df46a5da878e P b9f8880aef354e3daf773efd8c99ad5c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 650f778bf77344779339041522b0028a (127.27.180.130:44747): Not found: Tablet not found: d6c21764809c449998d4df46a5da878e
I20260811 18:58:48.563547 27006 leader_election.cc:302] T d6c21764809c449998d4df46a5da878e P b9f8880aef354e3daf773efd8c99ad5c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b9f8880aef354e3daf773efd8c99ad5c; no voters: 650f778bf77344779339041522b0028a, d825edd376d9480f8142afe7eefab04f
I20260811 18:58:48.563812 27722 raft_consensus.cc:2726] T d6c21764809c449998d4df46a5da878e P b9f8880aef354e3daf773efd8c99ad5c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 18:58:49.116744 27731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:49.116316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33249 (local address 127.27.180.129:52401)
0811 18:58:49.116583 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:49.116642 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:33249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":79,"threads_started":1}
I20260811 18:58:49.258224 27734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:49.257842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33249 (local address 127.27.180.130:36905)
0811 18:58:49.258073 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:49.258129 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:33249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":76,"threads_started":1}
I20260811 18:58:49.435719 27738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:49.435299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33249 (local address 127.27.180.131:59061)
0811 18:58:49.435551 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:49.435607 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:33249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":76,"threads_started":1}
I20260811 18:58:50.117944 27740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:50.117498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33249 (local address 127.27.180.129:54405)
0811 18:58:50.117772 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:50.117839 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:33249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":78,"threads_started":1}
I20260811 18:58:50.259349 27743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:50.258920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33249 (local address 127.27.180.130:34375)
0811 18:58:50.259184 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:50.259240 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:33249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":91,"threads_started":1}
I20260811 18:58:50.436842 27745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:50.436431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33249 (local address 127.27.180.131:52881)
0811 18:58:50.436679 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:50.436737 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:33249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":72,"threads_started":1}
I20260811 18:58:50.492626 27746 raft_consensus.cc:482] T d6c21764809c449998d4df46a5da878e P b9f8880aef354e3daf773efd8c99ad5c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:58:50.492761 27746 raft_consensus.cc:504] T d6c21764809c449998d4df46a5da878e P b9f8880aef354e3daf773efd8c99ad5c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d825edd376d9480f8142afe7eefab04f" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 44763 } } peers { permanent_uuid: "650f778bf77344779339041522b0028a" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 44747 } } peers { permanent_uuid: "b9f8880aef354e3daf773efd8c99ad5c" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 40553 } }
I20260811 18:58:50.493202 27746 leader_election.cc:288] T d6c21764809c449998d4df46a5da878e P b9f8880aef354e3daf773efd8c99ad5c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d825edd376d9480f8142afe7eefab04f (127.27.180.131:44763), 650f778bf77344779339041522b0028a (127.27.180.130:44747)
W20260811 18:58:50.494022 27004 leader_election.cc:341] T d6c21764809c449998d4df46a5da878e P b9f8880aef354e3daf773efd8c99ad5c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d825edd376d9480f8142afe7eefab04f (127.27.180.131:44763): Not found: Tablet not found: d6c21764809c449998d4df46a5da878e
W20260811 18:58:50.494127 27006 leader_election.cc:341] T d6c21764809c449998d4df46a5da878e P b9f8880aef354e3daf773efd8c99ad5c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 650f778bf77344779339041522b0028a (127.27.180.130:44747): Not found: Tablet not found: d6c21764809c449998d4df46a5da878e
I20260811 18:58:50.494222 27006 leader_election.cc:302] T d6c21764809c449998d4df46a5da878e P b9f8880aef354e3daf773efd8c99ad5c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b9f8880aef354e3daf773efd8c99ad5c; no voters: 650f778bf77344779339041522b0028a, d825edd376d9480f8142afe7eefab04f
I20260811 18:58:50.494367 27746 raft_consensus.cc:2726] T d6c21764809c449998d4df46a5da878e P b9f8880aef354e3daf773efd8c99ad5c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 18:58:51.119200 27748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:51.118739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33249 (local address 127.27.180.129:58555)
0811 18:58:51.119031 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:51.119088 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:33249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":97,"threads_started":1}
I20260811 18:58:51.260416 27750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:51.260032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33249 (local address 127.27.180.130:50757)
0811 18:58:51.260265 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:51.260325 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:33249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":75,"threads_started":1}
I20260811 18:58:51.437994 27751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:51.437574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33249 (local address 127.27.180.131:58319)
0811 18:58:51.437837 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:51.437894 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:33249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":77,"threads_started":1}
I20260811 18:58:52.120625 27752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:52.120132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33249 (local address 127.27.180.129:41519)
0811 18:58:52.120462 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:52.120522 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:33249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":86,"threads_started":1}
I20260811 18:58:52.261643 27756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:52.261238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33249 (local address 127.27.180.130:38919)
0811 18:58:52.261488 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:52.261546 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:33249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":75,"threads_started":1}
I20260811 18:58:52.439141 27757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:52.438748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33249 (local address 127.27.180.131:52667)
0811 18:58:52.438992 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:52.439045 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:33249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":89,"threads_started":1}
I20260811 18:58:52.544235 27758 raft_consensus.cc:482] T d6c21764809c449998d4df46a5da878e P b9f8880aef354e3daf773efd8c99ad5c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:58:52.544390 27758 raft_consensus.cc:504] T d6c21764809c449998d4df46a5da878e P b9f8880aef354e3daf773efd8c99ad5c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d825edd376d9480f8142afe7eefab04f" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 44763 } } peers { permanent_uuid: "650f778bf77344779339041522b0028a" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 44747 } } peers { permanent_uuid: "b9f8880aef354e3daf773efd8c99ad5c" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 40553 } }
I20260811 18:58:52.544935 27758 leader_election.cc:288] T d6c21764809c449998d4df46a5da878e P b9f8880aef354e3daf773efd8c99ad5c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d825edd376d9480f8142afe7eefab04f (127.27.180.131:44763), 650f778bf77344779339041522b0028a (127.27.180.130:44747)
W20260811 18:58:52.545953 27006 leader_election.cc:341] T d6c21764809c449998d4df46a5da878e P b9f8880aef354e3daf773efd8c99ad5c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 650f778bf77344779339041522b0028a (127.27.180.130:44747): Not found: Tablet not found: d6c21764809c449998d4df46a5da878e
W20260811 18:58:52.546103 27004 leader_election.cc:341] T d6c21764809c449998d4df46a5da878e P b9f8880aef354e3daf773efd8c99ad5c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d825edd376d9480f8142afe7eefab04f (127.27.180.131:44763): Not found: Tablet not found: d6c21764809c449998d4df46a5da878e
I20260811 18:58:52.546196 27004 leader_election.cc:302] T d6c21764809c449998d4df46a5da878e P b9f8880aef354e3daf773efd8c99ad5c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b9f8880aef354e3daf773efd8c99ad5c; no voters: 650f778bf77344779339041522b0028a, d825edd376d9480f8142afe7eefab04f
I20260811 18:58:52.546394 27758 raft_consensus.cc:2726] T d6c21764809c449998d4df46a5da878e P b9f8880aef354e3daf773efd8c99ad5c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 18:58:53.122234 27759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:53.121613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33249 (local address 127.27.180.129:53569)
0811 18:58:53.121982 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:53.122078 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:33249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":109,"threads_started":1}
I20260811 18:58:53.264786 27760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:53.263272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33249 (local address 127.27.180.130:47925)
0811 18:58:53.264133 (+ 861us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:53.264458 (+ 325us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:33249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":594,"thread_start_us":250,"threads_started":1}
I20260811 18:58:53.440753 27761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:53.440062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33249 (local address 127.27.180.131:58315)
0811 18:58:53.440460 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:53.440596 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:33249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":137,"threads_started":1}
I20260811 18:58:54.123739 27762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:54.123231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33249 (local address 127.27.180.129:39571)
0811 18:58:54.123535 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:54.123613 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:33249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":84,"threads_started":1}
I20260811 18:58:54.266069 27763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:54.265556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33249 (local address 127.27.180.130:48563)
0811 18:58:54.265888 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:54.265968 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:33249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":78,"threads_started":1}
I20260811 18:58:54.442096 27764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:54.441588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33249 (local address 127.27.180.131:45337)
0811 18:58:54.441932 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:54.441994 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:33249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":93,"threads_started":1}
I20260811 18:58:54.914881 27765 raft_consensus.cc:482] T d6c21764809c449998d4df46a5da878e P b9f8880aef354e3daf773efd8c99ad5c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:58:54.915033 27765 raft_consensus.cc:504] T d6c21764809c449998d4df46a5da878e P b9f8880aef354e3daf773efd8c99ad5c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d825edd376d9480f8142afe7eefab04f" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 44763 } } peers { permanent_uuid: "650f778bf77344779339041522b0028a" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 44747 } } peers { permanent_uuid: "b9f8880aef354e3daf773efd8c99ad5c" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 40553 } }
I20260811 18:58:54.915468 27765 leader_election.cc:288] T d6c21764809c449998d4df46a5da878e P b9f8880aef354e3daf773efd8c99ad5c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d825edd376d9480f8142afe7eefab04f (127.27.180.131:44763), 650f778bf77344779339041522b0028a (127.27.180.130:44747)
W20260811 18:58:54.916265 27006 leader_election.cc:341] T d6c21764809c449998d4df46a5da878e P b9f8880aef354e3daf773efd8c99ad5c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 650f778bf77344779339041522b0028a (127.27.180.130:44747): Not found: Tablet not found: d6c21764809c449998d4df46a5da878e
W20260811 18:58:54.916390 27004 leader_election.cc:341] T d6c21764809c449998d4df46a5da878e P b9f8880aef354e3daf773efd8c99ad5c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d825edd376d9480f8142afe7eefab04f (127.27.180.131:44763): Not found: Tablet not found: d6c21764809c449998d4df46a5da878e
I20260811 18:58:54.916461 27004 leader_election.cc:302] T d6c21764809c449998d4df46a5da878e P b9f8880aef354e3daf773efd8c99ad5c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b9f8880aef354e3daf773efd8c99ad5c; no voters: 650f778bf77344779339041522b0028a, d825edd376d9480f8142afe7eefab04f
I20260811 18:58:54.916623 27765 raft_consensus.cc:2726] T d6c21764809c449998d4df46a5da878e P b9f8880aef354e3daf773efd8c99ad5c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 18:58:55.125021 27766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:55.124576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33249 (local address 127.27.180.129:45087)
0811 18:58:55.124856 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:55.124917 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:33249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":87,"threads_started":1}
I20260811 18:58:55.267247 27770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:55.266830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33249 (local address 127.27.180.130:39571)
0811 18:58:55.267095 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:55.267153 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:33249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":84,"threads_started":1}
I20260811 18:58:55.443279 27772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:55.442844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33249 (local address 127.27.180.131:49327)
0811 18:58:55.443122 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:55.443181 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:33249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":75,"threads_started":1}
I20260811 18:58:56.126335 27773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:56.125845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33249 (local address 127.27.180.129:42181)
0811 18:58:56.126167 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:56.126228 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:33249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":91,"threads_started":1}
I20260811 18:58:56.268322 27774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:56.267919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33249 (local address 127.27.180.130:50907)
0811 18:58:56.268166 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:56.268221 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:33249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":73,"threads_started":1}
I20260811 18:58:56.444540 27778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:56.444035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33249 (local address 127.27.180.131:34987)
0811 18:58:56.444369 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:56.444438 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:33249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":97,"threads_started":1}
I20260811 18:58:57.127664 27779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:57.127204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33249 (local address 127.27.180.129:56569)
0811 18:58:57.127504 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:57.127565 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:33249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":73,"threads_started":1}
I20260811 18:58:57.269491 27781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:57.269062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33249 (local address 127.27.180.130:35927)
0811 18:58:57.269338 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:57.269394 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:33249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":92,"threads_started":1}
I20260811 18:58:57.445712 27782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:57.445302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33249 (local address 127.27.180.131:36469)
0811 18:58:57.445557 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:57.445616 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:33249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":75,"threads_started":1}
I20260811 18:58:58.128795 27785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:58.128372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33249 (local address 127.27.180.129:40577)
0811 18:58:58.128638 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:58.128693 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:33249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":81,"threads_started":1}
I20260811 18:58:58.226660 27787 raft_consensus.cc:482] T d6c21764809c449998d4df46a5da878e P b9f8880aef354e3daf773efd8c99ad5c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:58:58.226828 27787 raft_consensus.cc:504] T d6c21764809c449998d4df46a5da878e P b9f8880aef354e3daf773efd8c99ad5c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d825edd376d9480f8142afe7eefab04f" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 44763 } } peers { permanent_uuid: "650f778bf77344779339041522b0028a" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 44747 } } peers { permanent_uuid: "b9f8880aef354e3daf773efd8c99ad5c" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 40553 } }
I20260811 18:58:58.227299 27787 leader_election.cc:288] T d6c21764809c449998d4df46a5da878e P b9f8880aef354e3daf773efd8c99ad5c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d825edd376d9480f8142afe7eefab04f (127.27.180.131:44763), 650f778bf77344779339041522b0028a (127.27.180.130:44747)
W20260811 18:58:58.228001 27004 leader_election.cc:341] T d6c21764809c449998d4df46a5da878e P b9f8880aef354e3daf773efd8c99ad5c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d825edd376d9480f8142afe7eefab04f (127.27.180.131:44763): Not found: Tablet not found: d6c21764809c449998d4df46a5da878e
W20260811 18:58:58.228089 27006 leader_election.cc:341] T d6c21764809c449998d4df46a5da878e P b9f8880aef354e3daf773efd8c99ad5c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 650f778bf77344779339041522b0028a (127.27.180.130:44747): Not found: Tablet not found: d6c21764809c449998d4df46a5da878e
I20260811 18:58:58.228205 27006 leader_election.cc:302] T d6c21764809c449998d4df46a5da878e P b9f8880aef354e3daf773efd8c99ad5c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b9f8880aef354e3daf773efd8c99ad5c; no voters: 650f778bf77344779339041522b0028a, d825edd376d9480f8142afe7eefab04f
I20260811 18:58:58.228367 27787 raft_consensus.cc:2726] T d6c21764809c449998d4df46a5da878e P b9f8880aef354e3daf773efd8c99ad5c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 18:58:58.270509 27788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:58.270110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33249 (local address 127.27.180.130:34933)
0811 18:58:58.270361 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:58.270415 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:33249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":71,"threads_started":1}
I20260811 18:58:58.446802 27790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:58.446379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33249 (local address 127.27.180.131:34203)
0811 18:58:58.446649 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:58.446705 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:33249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":92,"threads_started":1}
I20260811 18:58:59.129951 27793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:59.129516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33249 (local address 127.27.180.129:46277)
0811 18:58:59.129778 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:59.129836 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:33249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":77,"threads_started":1}
I20260811 18:58:59.271623 27798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:59.271233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33249 (local address 127.27.180.130:48097)
0811 18:58:59.271469 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:59.271524 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:33249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":95,"threads_started":1}
I20260811 18:58:59.448032 27799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:58:59.447582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33249 (local address 127.27.180.131:43917)
0811 18:58:59.447862 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0811 18:58:59.447921 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:33249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":80,"threads_started":1}
I20260811 18:59:00.131086 27800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:00.130677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33249 (local address 127.27.180.129:56979)
0811 18:59:00.130920 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:00.130977 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:33249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":72,"threads_started":1}
I20260811 18:59:00.272672 27801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:00.272292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33249 (local address 127.27.180.130:38739)
0811 18:59:00.272527 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:00.272579 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:33249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":80,"threads_started":1}
I20260811 18:59:00.449257 27802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:00.448816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33249 (local address 127.27.180.131:59899)
0811 18:59:00.449068 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:00.449155 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:33249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":81,"threads_started":1}
I20260811 18:59:01.002887 27804 raft_consensus.cc:482] T d6c21764809c449998d4df46a5da878e P b9f8880aef354e3daf773efd8c99ad5c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:59:01.003044 27804 raft_consensus.cc:504] T d6c21764809c449998d4df46a5da878e P b9f8880aef354e3daf773efd8c99ad5c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d825edd376d9480f8142afe7eefab04f" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 44763 } } peers { permanent_uuid: "650f778bf77344779339041522b0028a" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 44747 } } peers { permanent_uuid: "b9f8880aef354e3daf773efd8c99ad5c" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 40553 } }
I20260811 18:59:01.003510 27804 leader_election.cc:288] T d6c21764809c449998d4df46a5da878e P b9f8880aef354e3daf773efd8c99ad5c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d825edd376d9480f8142afe7eefab04f (127.27.180.131:44763), 650f778bf77344779339041522b0028a (127.27.180.130:44747)
W20260811 18:59:01.004480 27004 leader_election.cc:341] T d6c21764809c449998d4df46a5da878e P b9f8880aef354e3daf773efd8c99ad5c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d825edd376d9480f8142afe7eefab04f (127.27.180.131:44763): Not found: Tablet not found: d6c21764809c449998d4df46a5da878e
W20260811 18:59:01.004609 27006 leader_election.cc:341] T d6c21764809c449998d4df46a5da878e P b9f8880aef354e3daf773efd8c99ad5c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 650f778bf77344779339041522b0028a (127.27.180.130:44747): Not found: Tablet not found: d6c21764809c449998d4df46a5da878e
I20260811 18:59:01.004717 27006 leader_election.cc:302] T d6c21764809c449998d4df46a5da878e P b9f8880aef354e3daf773efd8c99ad5c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b9f8880aef354e3daf773efd8c99ad5c; no voters: 650f778bf77344779339041522b0028a, d825edd376d9480f8142afe7eefab04f
I20260811 18:59:01.004889 27804 raft_consensus.cc:2726] T d6c21764809c449998d4df46a5da878e P b9f8880aef354e3daf773efd8c99ad5c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 18:59:01.132190 27805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:01.131806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33249 (local address 127.27.180.129:44817)
0811 18:59:01.132038 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:01.132091 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:33249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":76,"threads_started":1}
I20260811 18:59:01.273764 27806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:01.273373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33249 (local address 127.27.180.130:60335)
0811 18:59:01.273617 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:01.273671 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:33249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":75,"threads_started":1}
I20260811 18:59:01.450521 27807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:01.450040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33249 (local address 127.27.180.131:41971)
0811 18:59:01.450352 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:01.450418 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:33249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":83,"threads_started":1}
I20260811 18:59:02.133545 27808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:02.133117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33249 (local address 127.27.180.129:46791)
0811 18:59:02.133392 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:02.133447 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:33249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":73,"threads_started":1}
I20260811 18:59:02.274953 27810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:02.274526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33249 (local address 127.27.180.130:52293)
0811 18:59:02.274804 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:02.274860 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:33249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":75,"threads_started":1}
I20260811 18:59:02.439306 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 26904
I20260811 18:59:02.457255 27811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:02.455903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33249 (local address 127.27.180.131:55521)
0811 18:59:02.457051 (+ 1148us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:02.457138 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.190:33249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1056,"thread_start_us":518,"threads_started":1}
I20260811 18:59:02.463024 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 27225
I20260811 18:59:02.475643 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 27468
2026-08-11T18:59:02Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59562 ms)
[----------] 30 tests from SecurityITest (246282 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474496157106-28370-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:59:02 dist-test-slave-2j7r krb5kdc[27816](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:59:02 dist-test-slave-2j7r krb5kdc[27816](info): set up 2 sockets
Aug 11 18:59:02 dist-test-slave-2j7r krb5kdc[27816](info): commencing operation
krb5kdc: starting...
W20260811 18:59:03.713471 28370 mini_kdc.cc:121] Time spent starting KDC: real 1.210s user 0.003s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474496157106-28370-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474496157106-28370-0/krb5kdc/test-user.keytab.
Aug 11 18:59:03 dist-test-slave-2j7r krb5kdc[27816](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474743, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:59:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:59:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.180.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.180.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.180.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab.
Entry for principal kudu/127.27.180.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab.
I20260811 18:59:03.832993 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474496157106-28370-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474496157106-28370-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474496157106-28370-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.190:42917
--webserver_interface=127.27.180.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab
--principal=kudu/127.27.180.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:41179
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.180.190:42917
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:59:03.943130 27836 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:59:03.943398 27836 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:59:03.943436 27836 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:59:03.947466 27836 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:59:03.947522 27836 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:59:03.947541 27836 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:59:03.947556 27836 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:59:03.947570 27836 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:59:03.952889 27836 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:41179
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474496157106-28370-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474496157106-28370-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.180.190:42917
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab
--principal=kudu/127.27.180.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.190:42917
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.180.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27836
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474496157106-28370-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:59:03.953948 27836 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:59:03.954946 27836 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:59:03.964203 27853 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:03.964264 27845 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:03.964309 27846 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:59:03.964383 27836 server_base.cc:1034] running on GCE node
I20260811 18:59:03.965029 27836 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:59:03.966145 27836 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:59:03.967298 27836 hybrid_clock.cc:648] HybridClock initialized: now 1786474743967282 us; error 28 us; skew 500 ppm
Aug 11 18:59:03 dist-test-slave-2j7r krb5kdc[27816](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474743, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:59:03.970089 27836 init.cc:377] Logged in from keytab as kudu/127.27.180.190@KRBTEST.COM (short username kudu)
I20260811 18:59:03.970956 27836 webserver.cc:460] Webserver started at http://127.27.180.190:45245/ using document root <none> and password file <none>
I20260811 18:59:03.971496 27836 fs_manager.cc:359] Metadata directory not provided
I20260811 18:59:03.971556 27836 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:59:03.971748 27836 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:59:03.973959 27836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474496157106-28370-0/minicluster-data/master-0/data/instance:
uuid: "6e52c276e0c64b8abfbdf33c1df0d141"
format_stamp: "Formatted at 2026-08-11 18:59:03 on dist-test-slave-2j7r"
server_key: "b97850b60b8a48e9abf659497817ec13"
server_key_iv: "fe7593ac7d4dec79a9718aa4e3978667"
server_key_version: "encryptionkey@0"
I20260811 18:59:03.974510 27836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474496157106-28370-0/minicluster-data/master-0/wal/instance:
uuid: "6e52c276e0c64b8abfbdf33c1df0d141"
format_stamp: "Formatted at 2026-08-11 18:59:03 on dist-test-slave-2j7r"
server_key: "b97850b60b8a48e9abf659497817ec13"
server_key_iv: "fe7593ac7d4dec79a9718aa4e3978667"
server_key_version: "encryptionkey@0"
I20260811 18:59:03.978180 27836 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260811 18:59:03.980605 27863 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:03.981873 27836 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 18:59:03.981983 27836 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474496157106-28370-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474496157106-28370-0/minicluster-data/master-0/wal
uuid: "6e52c276e0c64b8abfbdf33c1df0d141"
format_stamp: "Formatted at 2026-08-11 18:59:03 on dist-test-slave-2j7r"
server_key: "b97850b60b8a48e9abf659497817ec13"
server_key_iv: "fe7593ac7d4dec79a9718aa4e3978667"
server_key_version: "encryptionkey@0"
I20260811 18:59:03.982095 27836 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474496157106-28370-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474496157106-28370-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474496157106-28370-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:59:04.018285 27836 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:59:04.021638 27836 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:59:04.021836 27836 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:59:04.028625 27836 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.190:42917
I20260811 18:59:04.028672 27968 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.190:42917 every 8 connection(s)
I20260811 18:59:04.029580 27836 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
I20260811 18:59:04.032342 27970 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:59:04.037873 27970 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e52c276e0c64b8abfbdf33c1df0d141: Bootstrap starting.
I20260811 18:59:04.039886 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 27836
I20260811 18:59:04.040030 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474496157106-28370-0/minicluster-data/master-0/wal/instance
I20260811 18:59:04.040364 28370 external_mini_cluster.cc:1428] Setting key 93527a9c21a062c381dc7363523dc639
I20260811 18:59:04.040586 27970 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6e52c276e0c64b8abfbdf33c1df0d141: Neither blocks nor log segments found. Creating new log.
I20260811 18:59:04.041483 27970 log.cc:824] T 00000000000000000000000000000000 P 6e52c276e0c64b8abfbdf33c1df0d141: Log is configured to *not* fsync() on all Append() calls
I20260811 18:59:04.043915 27970 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e52c276e0c64b8abfbdf33c1df0d141: No bootstrap required, opened a new log
I20260811 18:59:04.048439 27970 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6e52c276e0c64b8abfbdf33c1df0d141 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e52c276e0c64b8abfbdf33c1df0d141" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 42917 } }
I20260811 18:59:04.048748 27970 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6e52c276e0c64b8abfbdf33c1df0d141 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:59:04.048823 27970 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6e52c276e0c64b8abfbdf33c1df0d141 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e52c276e0c64b8abfbdf33c1df0d141, State: Initialized, Role: FOLLOWER
I20260811 18:59:04.049363 27970 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6e52c276e0c64b8abfbdf33c1df0d141 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e52c276e0c64b8abfbdf33c1df0d141" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 42917 } }
Aug 11 18:59:04 dist-test-slave-2j7r krb5kdc[27816](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474743, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM
I20260811 18:59:04.049540 27970 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6e52c276e0c64b8abfbdf33c1df0d141 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:59:04.049602 27970 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6e52c276e0c64b8abfbdf33c1df0d141 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:59:04.049711 27970 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6e52c276e0c64b8abfbdf33c1df0d141 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:59:04.050849 27970 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6e52c276e0c64b8abfbdf33c1df0d141 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e52c276e0c64b8abfbdf33c1df0d141" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 42917 } }
I20260811 18:59:04.051223 27970 leader_election.cc:302] T 00000000000000000000000000000000 P 6e52c276e0c64b8abfbdf33c1df0d141 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6e52c276e0c64b8abfbdf33c1df0d141; no voters:
I20260811 18:59:04.051612 27970 leader_election.cc:288] T 00000000000000000000000000000000 P 6e52c276e0c64b8abfbdf33c1df0d141 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:59:04.051760 27978 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6e52c276e0c64b8abfbdf33c1df0d141 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:59:04.052038 27978 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6e52c276e0c64b8abfbdf33c1df0d141 [term 1 LEADER]: Becoming Leader. State: Replica: 6e52c276e0c64b8abfbdf33c1df0d141, State: Running, Role: LEADER
I20260811 18:59:04.052811 27978 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6e52c276e0c64b8abfbdf33c1df0d141 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e52c276e0c64b8abfbdf33c1df0d141" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 42917 } }
I20260811 18:59:04.053241 27970 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6e52c276e0c64b8abfbdf33c1df0d141 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:59:04.054808 27980 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e52c276e0c64b8abfbdf33c1df0d141 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6e52c276e0c64b8abfbdf33c1df0d141. Latest consensus state: current_term: 1 leader_uuid: "6e52c276e0c64b8abfbdf33c1df0d141" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e52c276e0c64b8abfbdf33c1df0d141" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 42917 } } }
I20260811 18:59:04.055127 27980 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e52c276e0c64b8abfbdf33c1df0d141 [sys.catalog]: This master's current role is: LEADER
I20260811 18:59:04.055365 27979 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e52c276e0c64b8abfbdf33c1df0d141 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6e52c276e0c64b8abfbdf33c1df0d141" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e52c276e0c64b8abfbdf33c1df0d141" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 42917 } } }
I20260811 18:59:04.055481 27979 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e52c276e0c64b8abfbdf33c1df0d141 [sys.catalog]: This master's current role is: LEADER
I20260811 18:59:04.055771 27989 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:59:04.056406 27973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:04.041607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41946 (local address 127.27.180.190:42917)
0811 18:59:04.042033 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:04.042049 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:04.042084 (+ 35us) server_negotiation.cc:407] Connection header received
0811 18:59:04.042726 (+ 642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:04.042743 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:04.043062 (+ 319us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:04.043341 (+ 279us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:04.044269 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:04.045273 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:04.046559 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:04.047071 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:04.047211 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:04.050233 (+ 3022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:04.050239 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:04.050252 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:59:04.050281 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:04.052281 (+ 2000us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:04.052838 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:04.052844 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:04.052863 (+ 19us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:04.053012 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:04.053381 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:04.053384 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:04.053386 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:04.053929 (+ 543us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:04.054864 (+ 935us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:04.055037 (+ 173us) server_negotiation.cc:299] Negotiation successful
0811 18:59:04.055375 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":125,"threads_started":1}
I20260811 18:59:04.060469 27989 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:59:04.069280 27989 catalog_manager.cc:1347] Generated new cluster ID: f861b505a05b44a9b01db2efe77cfe33
I20260811 18:59:04.069366 27989 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:59:04.094671 27989 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:59:04.095917 27989 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:59:04.102200 27989 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6e52c276e0c64b8abfbdf33c1df0d141: Generated new TSK 0
I20260811 18:59:04.103047 27989 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.180.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.180.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.180.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.129.keytab.
Entry for principal kudu/127.27.180.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.129.keytab.
I20260811 18:59:04.137429 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474496157106-28370-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474496157106-28370-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474496157106-28370-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474496157106-28370-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.180.129:0
--local_ip_for_outbound_sockets=127.27.180.129
--webserver_interface=127.27.180.129
--webserver_port=0
--tserver_master_addrs=127.27.180.190:42917
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.129.keytab
--principal=kudu/127.27.180.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:41179
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:59:04.248183 28014 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:59:04.248472 28014 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:59:04.248517 28014 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:59:04.252509 28014 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:59:04.252575 28014 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:59:04.252635 28014 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.180.129
I20260811 18:59:04.258666 28014 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:41179
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474496157106-28370-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474496157106-28370-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.129.keytab
--principal=kudu/127.27.180.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.180.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474496157106-28370-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.180.129
--webserver_port=0
--tserver_master_addrs=127.27.180.190:42917
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28014
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.180.129
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474496157106-28370-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:59:04.259918 28014 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:59:04.260931 28014 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:59:04.270479 28032 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:04.270567 28025 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:04.270680 28027 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:59:04.271029 28014 server_base.cc:1034] running on GCE node
I20260811 18:59:04.271402 28014 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:59:04.271987 28014 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:59:04.273150 28014 hybrid_clock.cc:648] HybridClock initialized: now 1786474744273126 us; error 35 us; skew 500 ppm
Aug 11 18:59:04 dist-test-slave-2j7r krb5kdc[27816](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474744, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:59:04.275856 28014 init.cc:377] Logged in from keytab as kudu/127.27.180.129@KRBTEST.COM (short username kudu)
I20260811 18:59:04.276767 28014 webserver.cc:460] Webserver started at http://127.27.180.129:33617/ using document root <none> and password file <none>
I20260811 18:59:04.277364 28014 fs_manager.cc:359] Metadata directory not provided
I20260811 18:59:04.277429 28014 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:59:04.277621 28014 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:59:04.279687 28014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474496157106-28370-0/minicluster-data/ts-0/data/instance:
uuid: "07c141020dd24574b218d794f88c3fe1"
format_stamp: "Formatted at 2026-08-11 18:59:04 on dist-test-slave-2j7r"
server_key: "16b366ce27886d5ec0d8dc89f656fa49"
server_key_iv: "4c145a8857702acbbd144b1b4b0354d5"
server_key_version: "encryptionkey@0"
I20260811 18:59:04.280227 28014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474496157106-28370-0/minicluster-data/ts-0/wal/instance:
uuid: "07c141020dd24574b218d794f88c3fe1"
format_stamp: "Formatted at 2026-08-11 18:59:04 on dist-test-slave-2j7r"
server_key: "16b366ce27886d5ec0d8dc89f656fa49"
server_key_iv: "4c145a8857702acbbd144b1b4b0354d5"
server_key_version: "encryptionkey@0"
I20260811 18:59:04.284103 28014 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 18:59:04.286736 28044 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:04.287972 28014 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 18:59:04.288080 28014 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474496157106-28370-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474496157106-28370-0/minicluster-data/ts-0/wal
uuid: "07c141020dd24574b218d794f88c3fe1"
format_stamp: "Formatted at 2026-08-11 18:59:04 on dist-test-slave-2j7r"
server_key: "16b366ce27886d5ec0d8dc89f656fa49"
server_key_iv: "4c145a8857702acbbd144b1b4b0354d5"
server_key_version: "encryptionkey@0"
I20260811 18:59:04.288184 28014 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474496157106-28370-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474496157106-28370-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474496157106-28370-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:59:04.301204 28014 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:59:04.304105 28014 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:59:04.304275 28014 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:59:04.304850 28014 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:59:04.305702 28014 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:59:04.305747 28014 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:04.305800 28014 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:59:04.305830 28014 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:04.314198 28014 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.129:41313
I20260811 18:59:04.314262 28231 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.129:41313 every 8 connection(s)
I20260811 18:59:04.315057 28014 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474496157106-28370-0/minicluster-data/ts-0/data/info.pb
I20260811 18:59:04.323951 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 28014
I20260811 18:59:04.324126 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474496157106-28370-0/minicluster-data/ts-0/wal/instance
I20260811 18:59:04.324438 28370 external_mini_cluster.cc:1428] Setting key 3c994ce40da24774eaf2f6a3dc7cd063
Aug 11 18:59:04 dist-test-slave-2j7r krb5kdc[27816](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474744, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.129@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM
I20260811 18:59:04.328462 27973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:04.316702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.129:34731 (local address 127.27.180.190:42917)
0811 18:59:04.316853 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:04.316857 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:04.317660 (+ 803us) server_negotiation.cc:407] Connection header received
0811 18:59:04.318960 (+ 1300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:04.318963 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:04.319020 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:04.319095 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:04.320254 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:04.320910 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:04.322278 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:04.322514 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:04.322603 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:04.325186 (+ 2583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:04.325189 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:04.325191 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:59:04.325218 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:04.326553 (+ 1335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:04.327033 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:04.327036 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:04.327037 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:04.327113 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:04.327436 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:04.327438 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:04.327439 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:04.327608 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:04.327691 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:04.328173 (+ 482us) server_negotiation.cc:299] Negotiation successful
0811 18:59:04.328303 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 18:59:04.329478 28234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:04.317052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:42917 (local address 127.27.180.129:34731)
0811 18:59:04.317517 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:04.317558 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 18:59:04.318751 (+ 1193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:59:04.319263 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:59:04.319271 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:59:04.319632 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:59:04.320109 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:04.320122 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:04.321038 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:04.321041 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:04.322154 (+ 1113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:04.322164 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:04.322656 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:04.322660 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:04.322837 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:04.323367 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:59:04.323383 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:59:04.324979 (+ 1596us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:59:04.326661 (+ 1682us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:59:04.326667 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:59:04.326682 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:59:04.326931 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:59:04.327192 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:59:04.327195 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:59:04.327197 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:59:04.327340 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:59:04.327704 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:59:04.327711 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:59:04.327939 (+ 228us) client_negotiation.cc:769] Sending connection context
0811 18:59:04.328094 (+ 155us) client_negotiation.cc:240] Negotiation successful
0811 18:59:04.328418 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":116,"threads_started":1}
I20260811 18:59:04.330825 28232 heartbeater.cc:347] Connected to a master server at 127.27.180.190:42917
I20260811 18:59:04.331048 28232 heartbeater.cc:464] Registering TS with master...
I20260811 18:59:04.331523 28232 heartbeater.cc:511] Master 127.27.180.190:42917 requested a full tablet report, sending...
I20260811 18:59:04.333016 27891 ts_manager.cc:194] Registered new tserver with Master: 07c141020dd24574b218d794f88c3fe1 (127.27.180.129:41313)
I20260811 18:59:04.334129 27891 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.180.129@KRBTEST.COM'} at 127.27.180.129:34731
WARNING: no policy specified for kudu/127.27.180.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.180.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.180.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.130.keytab.
Entry for principal kudu/127.27.180.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.130.keytab.
I20260811 18:59:04.352736 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474496157106-28370-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474496157106-28370-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474496157106-28370-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474496157106-28370-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.180.130:0
--local_ip_for_outbound_sockets=127.27.180.130
--webserver_interface=127.27.180.130
--webserver_port=0
--tserver_master_addrs=127.27.180.190:42917
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.130.keytab
--principal=kudu/127.27.180.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:41179
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:59:04.463459 28240 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:59:04.463719 28240 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:59:04.463780 28240 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:59:04.467801 28240 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:59:04.467869 28240 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:59:04.467952 28240 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.180.130
I20260811 18:59:04.473572 28240 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:41179
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474496157106-28370-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474496157106-28370-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.130.keytab
--principal=kudu/127.27.180.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.180.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474496157106-28370-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.180.130
--webserver_port=0
--tserver_master_addrs=127.27.180.190:42917
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28240
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.180.130
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474496157106-28370-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:59:04.474614 28240 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:59:04.475621 28240 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:59:04.485276 28262 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:04.485284 28263 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:59:04.485479 28240 server_base.cc:1034] running on GCE node
W20260811 18:59:04.485632 28265 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:59:04.486044 28240 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:59:04.486754 28240 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:59:04.487922 28240 hybrid_clock.cc:648] HybridClock initialized: now 1786474744487905 us; error 33 us; skew 500 ppm
Aug 11 18:59:04 dist-test-slave-2j7r krb5kdc[27816](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474744, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:59:04.490736 28240 init.cc:377] Logged in from keytab as kudu/127.27.180.130@KRBTEST.COM (short username kudu)
I20260811 18:59:04.491777 28240 webserver.cc:460] Webserver started at http://127.27.180.130:35545/ using document root <none> and password file <none>
I20260811 18:59:04.492389 28240 fs_manager.cc:359] Metadata directory not provided
I20260811 18:59:04.492461 28240 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:59:04.492677 28240 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:59:04.494776 28240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474496157106-28370-0/minicluster-data/ts-1/data/instance:
uuid: "160bce60bb1e4264a56aae68be947c3b"
format_stamp: "Formatted at 2026-08-11 18:59:04 on dist-test-slave-2j7r"
server_key: "ed664fc11e2c05ee22389041a0d99f7e"
server_key_iv: "648c6bbde7539f433343aba9adba29d7"
server_key_version: "encryptionkey@0"
I20260811 18:59:04.495307 28240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474496157106-28370-0/minicluster-data/ts-1/wal/instance:
uuid: "160bce60bb1e4264a56aae68be947c3b"
format_stamp: "Formatted at 2026-08-11 18:59:04 on dist-test-slave-2j7r"
server_key: "ed664fc11e2c05ee22389041a0d99f7e"
server_key_iv: "648c6bbde7539f433343aba9adba29d7"
server_key_version: "encryptionkey@0"
I20260811 18:59:04.499069 28240 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260811 18:59:04.501498 28275 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:04.502738 28240 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 18:59:04.502844 28240 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474496157106-28370-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474496157106-28370-0/minicluster-data/ts-1/wal
uuid: "160bce60bb1e4264a56aae68be947c3b"
format_stamp: "Formatted at 2026-08-11 18:59:04 on dist-test-slave-2j7r"
server_key: "ed664fc11e2c05ee22389041a0d99f7e"
server_key_iv: "648c6bbde7539f433343aba9adba29d7"
server_key_version: "encryptionkey@0"
I20260811 18:59:04.502944 28240 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474496157106-28370-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474496157106-28370-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474496157106-28370-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:59:04.516206 28240 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:59:04.519333 28240 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:59:04.519543 28240 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:59:04.520136 28240 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:59:04.520998 28240 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:59:04.521045 28240 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:04.521116 28240 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:59:04.521147 28240 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:04.529757 28240 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.130:39085
I20260811 18:59:04.529791 28498 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.130:39085 every 8 connection(s)
I20260811 18:59:04.530596 28240 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474496157106-28370-0/minicluster-data/ts-1/data/info.pb
Aug 11 18:59:04 dist-test-slave-2j7r krb5kdc[27816](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474744, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.130@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM
I20260811 18:59:04.539902 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 28240
I20260811 18:59:04.540048 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474496157106-28370-0/minicluster-data/ts-1/wal/instance
I20260811 18:59:04.540360 28370 external_mini_cluster.cc:1428] Setting key c74c65eb34062fc40812ba6b8af3b554
I20260811 18:59:04.543503 27973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:04.532479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.130:51731 (local address 127.27.180.190:42917)
0811 18:59:04.532658 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:04.532663 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:04.533421 (+ 758us) server_negotiation.cc:407] Connection header received
0811 18:59:04.534693 (+ 1272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:04.534696 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:04.534746 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:04.534820 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:04.535928 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:04.536458 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:04.537556 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:04.537769 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:04.537850 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:04.540193 (+ 2343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:04.540196 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:04.540198 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:59:04.540222 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:04.541589 (+ 1367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:04.542062 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:04.542066 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:04.542068 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:04.542146 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:04.542480 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:04.542483 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:04.542485 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:04.542661 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:04.542747 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:04.543227 (+ 480us) server_negotiation.cc:299] Negotiation successful
0811 18:59:04.543347 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 18:59:04.544274 28502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:04.532796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:42917 (local address 127.27.180.130:51731)
0811 18:59:04.533276 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:04.533319 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 18:59:04.534505 (+ 1186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:59:04.534983 (+ 478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:59:04.534989 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:59:04.535333 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:59:04.535789 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:04.535802 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:04.536548 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:04.536551 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:04.537469 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:04.537479 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:04.537898 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:04.537901 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:04.538029 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:04.538561 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:59:04.538577 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:59:04.540036 (+ 1459us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:59:04.541698 (+ 1662us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:59:04.541704 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:59:04.541718 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:59:04.541965 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:59:04.542228 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:59:04.542231 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:59:04.542233 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:59:04.542389 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:59:04.542761 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:59:04.542768 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:59:04.543003 (+ 235us) client_negotiation.cc:769] Sending connection context
0811 18:59:04.543166 (+ 163us) client_negotiation.cc:240] Negotiation successful
0811 18:59:04.543496 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":105,"threads_started":1}
I20260811 18:59:04.545466 28499 heartbeater.cc:347] Connected to a master server at 127.27.180.190:42917
I20260811 18:59:04.545687 28499 heartbeater.cc:464] Registering TS with master...
I20260811 18:59:04.546161 28499 heartbeater.cc:511] Master 127.27.180.190:42917 requested a full tablet report, sending...
I20260811 18:59:04.547190 27891 ts_manager.cc:194] Registered new tserver with Master: 160bce60bb1e4264a56aae68be947c3b (127.27.180.130:39085)
I20260811 18:59:04.547945 27891 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.180.130@KRBTEST.COM'} at 127.27.180.130:51731
WARNING: no policy specified for kudu/127.27.180.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.180.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.180.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.131.keytab.
Entry for principal kudu/127.27.180.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.131.keytab.
I20260811 18:59:04.568554 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474496157106-28370-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474496157106-28370-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474496157106-28370-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474496157106-28370-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.180.131:0
--local_ip_for_outbound_sockets=127.27.180.131
--webserver_interface=127.27.180.131
--webserver_port=0
--tserver_master_addrs=127.27.180.190:42917
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.131.keytab
--principal=kudu/127.27.180.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:41179
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:59:04.685259 28507 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:59:04.685566 28507 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:59:04.685635 28507 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:59:04.689725 28507 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:59:04.689779 28507 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:59:04.689838 28507 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.180.131
I20260811 18:59:04.695025 28507 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:41179
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474496157106-28370-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474496157106-28370-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.131.keytab
--principal=kudu/127.27.180.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.180.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474496157106-28370-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.180.131
--webserver_port=0
--tserver_master_addrs=127.27.180.190:42917
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28507
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.180.131
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474496157106-28370-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:59:04.696039 28507 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:59:04.697010 28507 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:59:04.706379 28513 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:59:04.706506 28507 server_base.cc:1034] running on GCE node
W20260811 18:59:04.706459 28515 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:04.706408 28519 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:59:04.706993 28507 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:59:04.707566 28507 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:59:04.708751 28507 hybrid_clock.cc:648] HybridClock initialized: now 1786474744708717 us; error 49 us; skew 500 ppm
Aug 11 18:59:04 dist-test-slave-2j7r krb5kdc[27816](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474744, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:59:04.711545 28507 init.cc:377] Logged in from keytab as kudu/127.27.180.131@KRBTEST.COM (short username kudu)
I20260811 18:59:04.712440 28507 webserver.cc:460] Webserver started at http://127.27.180.131:40933/ using document root <none> and password file <none>
I20260811 18:59:04.712954 28507 fs_manager.cc:359] Metadata directory not provided
I20260811 18:59:04.713020 28507 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:59:04.713281 28507 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:59:04.715242 28507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474496157106-28370-0/minicluster-data/ts-2/data/instance:
uuid: "b09217442e1b4e78bf45c214855f9a2e"
format_stamp: "Formatted at 2026-08-11 18:59:04 on dist-test-slave-2j7r"
server_key: "7aa71a74403ad1ad30301c2013811027"
server_key_iv: "f29c18c5f2a4ed477967e64d04a209f7"
server_key_version: "encryptionkey@0"
I20260811 18:59:04.715785 28507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474496157106-28370-0/minicluster-data/ts-2/wal/instance:
uuid: "b09217442e1b4e78bf45c214855f9a2e"
format_stamp: "Formatted at 2026-08-11 18:59:04 on dist-test-slave-2j7r"
server_key: "7aa71a74403ad1ad30301c2013811027"
server_key_iv: "f29c18c5f2a4ed477967e64d04a209f7"
server_key_version: "encryptionkey@0"
I20260811 18:59:04.719345 28507 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 18:59:04.721668 28529 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:04.722863 28507 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 18:59:04.722970 28507 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474496157106-28370-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474496157106-28370-0/minicluster-data/ts-2/wal
uuid: "b09217442e1b4e78bf45c214855f9a2e"
format_stamp: "Formatted at 2026-08-11 18:59:04 on dist-test-slave-2j7r"
server_key: "7aa71a74403ad1ad30301c2013811027"
server_key_iv: "f29c18c5f2a4ed477967e64d04a209f7"
server_key_version: "encryptionkey@0"
I20260811 18:59:04.723065 28507 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474496157106-28370-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474496157106-28370-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474496157106-28370-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:59:04.768204 28507 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:59:04.771565 28507 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:59:04.771754 28507 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:59:04.772411 28507 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:59:04.773327 28507 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:59:04.773378 28507 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:04.773432 28507 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:59:04.773459 28507 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:04.782042 28507 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.131:44951
I20260811 18:59:04.782075 28753 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.131:44951 every 8 connection(s)
I20260811 18:59:04.782917 28507 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474496157106-28370-0/minicluster-data/ts-2/data/info.pb
I20260811 18:59:04.786854 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 28507
I20260811 18:59:04.786988 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474496157106-28370-0/minicluster-data/ts-2/wal/instance
I20260811 18:59:04.787366 28370 external_mini_cluster.cc:1428] Setting key 508d305e6a10fb871a1a360a39ab3a0d
Aug 11 18:59:04 dist-test-slave-2j7r krb5kdc[27816](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474744, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.131@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM
I20260811 18:59:04.796444 27973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:04.784706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.131:44123 (local address 127.27.180.190:42917)
0811 18:59:04.784837 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:04.784840 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:04.785693 (+ 853us) server_negotiation.cc:407] Connection header received
0811 18:59:04.787118 (+ 1425us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:04.787122 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:04.787180 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:04.787278 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:04.788589 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:04.789340 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:04.790454 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:04.790673 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:04.790760 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:04.793376 (+ 2616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:04.793379 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:04.793381 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:59:04.793407 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:04.794563 (+ 1156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:04.795027 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:04.795031 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:04.795033 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:04.795109 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:04.795434 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:04.795437 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:04.795439 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:04.795616 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:04.795702 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:04.796170 (+ 468us) server_negotiation.cc:299] Negotiation successful
0811 18:59:04.796286 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 18:59:04.797204 28760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:04.785070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:42917 (local address 127.27.180.131:44123)
0811 18:59:04.785548 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:04.785588 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 18:59:04.786876 (+ 1288us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:59:04.787491 (+ 615us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:59:04.787500 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:59:04.787869 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:59:04.788446 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:04.788462 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:04.789468 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:04.789470 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:04.790322 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:04.790331 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:04.790809 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:04.790814 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:04.790975 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:04.791601 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:59:04.791617 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:59:04.793199 (+ 1582us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:59:04.794672 (+ 1473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:59:04.794678 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:59:04.794692 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:59:04.794932 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:59:04.795191 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:59:04.795194 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:59:04.795196 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:59:04.795338 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:59:04.795715 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:59:04.795722 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:59:04.795946 (+ 224us) client_negotiation.cc:769] Sending connection context
0811 18:59:04.796106 (+ 160us) client_negotiation.cc:240] Negotiation successful
0811 18:59:04.796416 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":114,"threads_started":1}
I20260811 18:59:04.798218 28755 heartbeater.cc:347] Connected to a master server at 127.27.180.190:42917
I20260811 18:59:04.798432 28755 heartbeater.cc:464] Registering TS with master...
I20260811 18:59:04.798856 28755 heartbeater.cc:511] Master 127.27.180.190:42917 requested a full tablet report, sending...
I20260811 18:59:04.799774 27891 ts_manager.cc:194] Registered new tserver with Master: b09217442e1b4e78bf45c214855f9a2e (127.27.180.131:44951)
I20260811 18:59:04.800382 27891 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.180.131@KRBTEST.COM'} at 127.27.180.131:44123
I20260811 18:59:04.801011 28370 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 18:59:04.809410 27973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:04.802931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41956 (local address 127.27.180.190:42917)
0811 18:59:04.803048 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:04.803052 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:04.803239 (+ 187us) server_negotiation.cc:407] Connection header received
0811 18:59:04.803336 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:04.803338 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:04.803374 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:04.803436 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:04.804060 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:04.804559 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:04.805410 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:04.805619 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:04.805697 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:04.806848 (+ 1151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:04.806851 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:04.806853 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:59:04.806874 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:04.808084 (+ 1210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:04.808438 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:04.808440 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:04.808441 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:04.808504 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:04.808682 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:04.808684 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:04.808685 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:04.808868 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:04.808963 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:04.809148 (+ 185us) server_negotiation.cc:299] Negotiation successful
0811 18:59:04.809265 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 18:59:04.811910 27891 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41956:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:59:04.814885 27891 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:59:04.828590 28789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:04.822555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.129:41313 (local address 127.0.0.1:45204)
0811 18:59:04.823493 (+ 938us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:04.823541 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 18:59:04.823666 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:59:04.824193 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:59:04.824199 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:59:04.824221 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:59:04.824340 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:04.824347 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:04.825576 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:04.825580 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:04.826942 (+ 1362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:04.826954 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:04.828052 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:04.828056 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:04.828175 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:04.828211 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 18:59:04.828331 (+ 120us) client_negotiation.cc:240] Negotiation successful
0811 18:59:04.828447 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":758,"mutex_wait_us":4,"thread_start_us":76,"threads_started":1}
I20260811 18:59:04.828605 28787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:04.822858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45204 (local address 127.27.180.129:41313)
0811 18:59:04.823616 (+ 758us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:04.823631 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:04.823655 (+ 24us) server_negotiation.cc:407] Connection header received
0811 18:59:04.823827 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:04.823833 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:04.824035 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:04.824191 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:59:04.824445 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:04.825434 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:04.827074 (+ 1640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:04.827921 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:04.828046 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:04.828179 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:04.828364 (+ 185us) server_negotiation.cc:299] Negotiation successful
0811 18:59:04.828466 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":656,"thread_start_us":75,"threads_started":1}
I20260811 18:59:04.828778 28781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:04.822633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.131:44951 (local address 127.0.0.1:50312)
0811 18:59:04.823893 (+ 1260us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:04.823905 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 18:59:04.823989 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:59:04.824619 (+ 630us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:59:04.824622 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:59:04.824651 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:59:04.824846 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:04.824853 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:04.826054 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:04.826058 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:04.827192 (+ 1134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:04.827212 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:04.828402 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:04.828405 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:04.828492 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:04.828516 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 18:59:04.828581 (+ 65us) client_negotiation.cc:240] Negotiation successful
0811 18:59:04.828646 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1205,"mutex_wait_us":71,"spinlock_wait_cycles":6272,"thread_start_us":101,"threads_started":1}
I20260811 18:59:04.828995 28783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:04.822946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.130:39085 (local address 127.0.0.1:37650)
0811 18:59:04.823956 (+ 1010us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:04.823967 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 18:59:04.824039 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:59:04.824622 (+ 583us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:59:04.824625 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:59:04.824653 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:59:04.824868 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:04.824876 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:04.826050 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:04.826053 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:04.827238 (+ 1185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:04.827247 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:04.828320 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:04.828324 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:04.828403 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:04.828430 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 18:59:04.828493 (+ 63us) client_negotiation.cc:240] Negotiation successful
0811 18:59:04.828550 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":959,"mutex_wait_us":94,"spinlock_wait_cycles":5760,"thread_start_us":76,"threads_started":1}
I20260811 18:59:04.829172 28791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:04.823370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37650 (local address 127.27.180.130:39085)
0811 18:59:04.823675 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:04.823686 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:04.824123 (+ 437us) server_negotiation.cc:407] Connection header received
0811 18:59:04.824194 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:04.824198 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:04.824358 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:04.824479 (+ 121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:59:04.824986 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:04.825879 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:04.827350 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:04.828011 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:04.828128 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:04.828257 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:04.828660 (+ 403us) server_negotiation.cc:299] Negotiation successful
0811 18:59:04.829017 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":212,"thread_start_us":124,"threads_started":1}
I20260811 18:59:04.829316 28790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:04.823200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50312 (local address 127.27.180.131:44951)
0811 18:59:04.823827 (+ 627us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:04.823839 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:04.824075 (+ 236us) server_negotiation.cc:407] Connection header received
0811 18:59:04.824152 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:04.824157 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:04.824331 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:04.824477 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:59:04.824979 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:04.825942 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:04.827339 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:04.828123 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:04.828245 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:04.828339 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:04.829049 (+ 710us) server_negotiation.cc:299] Negotiation successful
0811 18:59:04.829177 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":536,"thread_start_us":136,"threads_started":1}
I20260811 18:59:04.831672 28365 tablet_service.cc:1467] Processing CreateTablet for tablet 918921d81f954bdbb708d8bd832bd16f (DEFAULT_TABLE table=test-table [id=fa3a2b90463d45078660050e40989249]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:59:04.831722 28123 tablet_service.cc:1467] Processing CreateTablet for tablet 918921d81f954bdbb708d8bd832bd16f (DEFAULT_TABLE table=test-table [id=fa3a2b90463d45078660050e40989249]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:59:04.831707 28612 tablet_service.cc:1467] Processing CreateTablet for tablet 918921d81f954bdbb708d8bd832bd16f (DEFAULT_TABLE table=test-table [id=fa3a2b90463d45078660050e40989249]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:59:04.832566 28365 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 918921d81f954bdbb708d8bd832bd16f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:59:04.832690 28612 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 918921d81f954bdbb708d8bd832bd16f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:59:04.832690 28123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 918921d81f954bdbb708d8bd832bd16f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:59:04.838805 28794 tablet_bootstrap.cc:492] T 918921d81f954bdbb708d8bd832bd16f P 160bce60bb1e4264a56aae68be947c3b: Bootstrap starting.
I20260811 18:59:04.840219 28796 tablet_bootstrap.cc:492] T 918921d81f954bdbb708d8bd832bd16f P 07c141020dd24574b218d794f88c3fe1: Bootstrap starting.
I20260811 18:59:04.840207 28798 tablet_bootstrap.cc:492] T 918921d81f954bdbb708d8bd832bd16f P b09217442e1b4e78bf45c214855f9a2e: Bootstrap starting.
I20260811 18:59:04.841249 28794 tablet_bootstrap.cc:654] T 918921d81f954bdbb708d8bd832bd16f P 160bce60bb1e4264a56aae68be947c3b: Neither blocks nor log segments found. Creating new log.
I20260811 18:59:04.841933 28796 tablet_bootstrap.cc:654] T 918921d81f954bdbb708d8bd832bd16f P 07c141020dd24574b218d794f88c3fe1: Neither blocks nor log segments found. Creating new log.
I20260811 18:59:04.842288 28798 tablet_bootstrap.cc:654] T 918921d81f954bdbb708d8bd832bd16f P b09217442e1b4e78bf45c214855f9a2e: Neither blocks nor log segments found. Creating new log.
I20260811 18:59:04.842521 28794 log.cc:824] T 918921d81f954bdbb708d8bd832bd16f P 160bce60bb1e4264a56aae68be947c3b: Log is configured to *not* fsync() on all Append() calls
I20260811 18:59:04.842613 28796 log.cc:824] T 918921d81f954bdbb708d8bd832bd16f P 07c141020dd24574b218d794f88c3fe1: Log is configured to *not* fsync() on all Append() calls
I20260811 18:59:04.843454 28798 log.cc:824] T 918921d81f954bdbb708d8bd832bd16f P b09217442e1b4e78bf45c214855f9a2e: Log is configured to *not* fsync() on all Append() calls
I20260811 18:59:04.844627 28794 tablet_bootstrap.cc:492] T 918921d81f954bdbb708d8bd832bd16f P 160bce60bb1e4264a56aae68be947c3b: No bootstrap required, opened a new log
I20260811 18:59:04.844698 28796 tablet_bootstrap.cc:492] T 918921d81f954bdbb708d8bd832bd16f P 07c141020dd24574b218d794f88c3fe1: No bootstrap required, opened a new log
I20260811 18:59:04.844834 28794 ts_tablet_manager.cc:1397] T 918921d81f954bdbb708d8bd832bd16f P 160bce60bb1e4264a56aae68be947c3b: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260811 18:59:04.844856 28796 ts_tablet_manager.cc:1397] T 918921d81f954bdbb708d8bd832bd16f P 07c141020dd24574b218d794f88c3fe1: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260811 18:59:04.845331 28798 tablet_bootstrap.cc:492] T 918921d81f954bdbb708d8bd832bd16f P b09217442e1b4e78bf45c214855f9a2e: No bootstrap required, opened a new log
I20260811 18:59:04.845494 28798 ts_tablet_manager.cc:1397] T 918921d81f954bdbb708d8bd832bd16f P b09217442e1b4e78bf45c214855f9a2e: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.004s
I20260811 18:59:04.848083 28796 raft_consensus.cc:348] T 918921d81f954bdbb708d8bd832bd16f P 07c141020dd24574b218d794f88c3fe1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07c141020dd24574b218d794f88c3fe1" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 41313 } } peers { permanent_uuid: "b09217442e1b4e78bf45c214855f9a2e" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 44951 } } peers { permanent_uuid: "160bce60bb1e4264a56aae68be947c3b" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 39085 } }
I20260811 18:59:04.848347 28796 raft_consensus.cc:374] T 918921d81f954bdbb708d8bd832bd16f P 07c141020dd24574b218d794f88c3fe1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:59:04.848405 28796 raft_consensus.cc:729] T 918921d81f954bdbb708d8bd832bd16f P 07c141020dd24574b218d794f88c3fe1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07c141020dd24574b218d794f88c3fe1, State: Initialized, Role: FOLLOWER
I20260811 18:59:04.848594 28794 raft_consensus.cc:348] T 918921d81f954bdbb708d8bd832bd16f P 160bce60bb1e4264a56aae68be947c3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07c141020dd24574b218d794f88c3fe1" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 41313 } } peers { permanent_uuid: "b09217442e1b4e78bf45c214855f9a2e" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 44951 } } peers { permanent_uuid: "160bce60bb1e4264a56aae68be947c3b" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 39085 } }
I20260811 18:59:04.848861 28794 raft_consensus.cc:374] T 918921d81f954bdbb708d8bd832bd16f P 160bce60bb1e4264a56aae68be947c3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:59:04.848814 28796 consensus_queue.cc:260] T 918921d81f954bdbb708d8bd832bd16f P 07c141020dd24574b218d794f88c3fe1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07c141020dd24574b218d794f88c3fe1" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 41313 } } peers { permanent_uuid: "b09217442e1b4e78bf45c214855f9a2e" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 44951 } } peers { permanent_uuid: "160bce60bb1e4264a56aae68be947c3b" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 39085 } }
I20260811 18:59:04.848922 28794 raft_consensus.cc:729] T 918921d81f954bdbb708d8bd832bd16f P 160bce60bb1e4264a56aae68be947c3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 160bce60bb1e4264a56aae68be947c3b, State: Initialized, Role: FOLLOWER
I20260811 18:59:04.849115 28798 raft_consensus.cc:348] T 918921d81f954bdbb708d8bd832bd16f P b09217442e1b4e78bf45c214855f9a2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07c141020dd24574b218d794f88c3fe1" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 41313 } } peers { permanent_uuid: "b09217442e1b4e78bf45c214855f9a2e" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 44951 } } peers { permanent_uuid: "160bce60bb1e4264a56aae68be947c3b" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 39085 } }
I20260811 18:59:04.849351 28798 raft_consensus.cc:374] T 918921d81f954bdbb708d8bd832bd16f P b09217442e1b4e78bf45c214855f9a2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:59:04.849406 28798 raft_consensus.cc:729] T 918921d81f954bdbb708d8bd832bd16f P b09217442e1b4e78bf45c214855f9a2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b09217442e1b4e78bf45c214855f9a2e, State: Initialized, Role: FOLLOWER
I20260811 18:59:04.849339 28794 consensus_queue.cc:260] T 918921d81f954bdbb708d8bd832bd16f P 160bce60bb1e4264a56aae68be947c3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07c141020dd24574b218d794f88c3fe1" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 41313 } } peers { permanent_uuid: "b09217442e1b4e78bf45c214855f9a2e" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 44951 } } peers { permanent_uuid: "160bce60bb1e4264a56aae68be947c3b" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 39085 } }
I20260811 18:59:04.849648 28232 heartbeater.cc:503] Master 127.27.180.190:42917 was elected leader, sending a full tablet report...
I20260811 18:59:04.849838 28796 ts_tablet_manager.cc:1428] T 918921d81f954bdbb708d8bd832bd16f P 07c141020dd24574b218d794f88c3fe1: Time spent starting tablet: real 0.005s user 0.003s sys 0.003s
I20260811 18:59:04.849838 28798 consensus_queue.cc:260] T 918921d81f954bdbb708d8bd832bd16f P b09217442e1b4e78bf45c214855f9a2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07c141020dd24574b218d794f88c3fe1" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 41313 } } peers { permanent_uuid: "b09217442e1b4e78bf45c214855f9a2e" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 44951 } } peers { permanent_uuid: "160bce60bb1e4264a56aae68be947c3b" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 39085 } }
I20260811 18:59:04.850274 28794 ts_tablet_manager.cc:1428] T 918921d81f954bdbb708d8bd832bd16f P 160bce60bb1e4264a56aae68be947c3b: Time spent starting tablet: real 0.005s user 0.007s sys 0.000s
I20260811 18:59:04.850631 28499 heartbeater.cc:503] Master 127.27.180.190:42917 was elected leader, sending a full tablet report...
I20260811 18:59:04.850888 28798 ts_tablet_manager.cc:1428] T 918921d81f954bdbb708d8bd832bd16f P b09217442e1b4e78bf45c214855f9a2e: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260811 18:59:04.850965 28755 heartbeater.cc:503] Master 127.27.180.190:42917 was elected leader, sending a full tablet report...
W20260811 18:59:05.031680 28500 tablet.cc:2369] T 918921d81f954bdbb708d8bd832bd16f P 160bce60bb1e4264a56aae68be947c3b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 18:59:05.033900 28756 tablet.cc:2369] T 918921d81f954bdbb708d8bd832bd16f P b09217442e1b4e78bf45c214855f9a2e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 18:59:05.066351 28233 tablet.cc:2369] T 918921d81f954bdbb708d8bd832bd16f P 07c141020dd24574b218d794f88c3fe1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:59:05.155910 28807 raft_consensus.cc:482] T 918921d81f954bdbb708d8bd832bd16f P b09217442e1b4e78bf45c214855f9a2e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:59:05.156118 28807 raft_consensus.cc:504] T 918921d81f954bdbb708d8bd832bd16f P b09217442e1b4e78bf45c214855f9a2e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07c141020dd24574b218d794f88c3fe1" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 41313 } } peers { permanent_uuid: "b09217442e1b4e78bf45c214855f9a2e" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 44951 } } peers { permanent_uuid: "160bce60bb1e4264a56aae68be947c3b" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 39085 } }
I20260811 18:59:05.157343 28807 leader_election.cc:288] T 918921d81f954bdbb708d8bd832bd16f P b09217442e1b4e78bf45c214855f9a2e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 07c141020dd24574b218d794f88c3fe1 (127.27.180.129:41313), 160bce60bb1e4264a56aae68be947c3b (127.27.180.130:39085)
I20260811 18:59:05.161291 28791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:05.157731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.131:57627 (local address 127.27.180.130:39085)
0811 18:59:05.157866 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:05.157872 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:05.157891 (+ 19us) server_negotiation.cc:407] Connection header received
0811 18:59:05.157952 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:05.157955 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:05.158012 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:05.158101 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:59:05.158425 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:05.159057 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:05.160238 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:05.160679 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:05.160771 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:05.160815 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:05.161071 (+ 256us) server_negotiation.cc:299] Negotiation successful
0811 18:59:05.161167 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 18:59:05.161293 28760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:05.157589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.130:39085 (local address 127.27.180.131:57627)
0811 18:59:05.157743 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:05.157759 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 18:59:05.157866 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:59:05.158119 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:59:05.158123 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:59:05.158143 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:59:05.158271 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:05.158278 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:05.159168 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:05.159172 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:05.160106 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:05.160116 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:05.160797 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:05.160801 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:05.160904 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:05.160973 (+ 69us) client_negotiation.cc:769] Sending connection context
0811 18:59:05.161036 (+ 63us) client_negotiation.cc:240] Negotiation successful
0811 18:59:05.161122 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59,"mutex_wait_us":54}
I20260811 18:59:05.161572 28787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:05.157742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.131:51041 (local address 127.27.180.129:41313)
0811 18:59:05.157878 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:05.157882 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:05.158292 (+ 410us) server_negotiation.cc:407] Connection header received
0811 18:59:05.158395 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:05.158397 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:05.158444 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:05.158503 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:59:05.158772 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:05.159297 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:05.160351 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:05.160781 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:05.160891 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:05.160940 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:05.161402 (+ 462us) server_negotiation.cc:299] Negotiation successful
0811 18:59:05.161485 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 18:59:05.161914 28809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:05.157589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.129:41313 (local address 127.27.180.131:51041)
0811 18:59:05.158186 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:05.158199 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:59:05.158311 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:59:05.158531 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:59:05.158534 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:59:05.158551 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:59:05.158673 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:05.158679 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:05.159402 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:05.159405 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:05.160221 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:05.160233 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:05.160906 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:05.160908 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:05.161013 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:05.161038 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 18:59:05.161710 (+ 672us) client_negotiation.cc:240] Negotiation successful
0811 18:59:05.161781 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":510,"mutex_wait_us":229,"thread_start_us":137,"threads_started":1}
I20260811 18:59:05.162130 28412 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "918921d81f954bdbb708d8bd832bd16f" candidate_uuid: "b09217442e1b4e78bf45c214855f9a2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "160bce60bb1e4264a56aae68be947c3b" is_pre_election: true
I20260811 18:59:05.162441 28412 raft_consensus.cc:2454] T 918921d81f954bdbb708d8bd832bd16f P 160bce60bb1e4264a56aae68be947c3b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b09217442e1b4e78bf45c214855f9a2e in term 0.
I20260811 18:59:05.162497 28160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "918921d81f954bdbb708d8bd832bd16f" candidate_uuid: "b09217442e1b4e78bf45c214855f9a2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07c141020dd24574b218d794f88c3fe1" is_pre_election: true
I20260811 18:59:05.162760 28160 raft_consensus.cc:2454] T 918921d81f954bdbb708d8bd832bd16f P 07c141020dd24574b218d794f88c3fe1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b09217442e1b4e78bf45c214855f9a2e in term 0.
I20260811 18:59:05.162889 28533 leader_election.cc:302] T 918921d81f954bdbb708d8bd832bd16f P b09217442e1b4e78bf45c214855f9a2e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 160bce60bb1e4264a56aae68be947c3b, b09217442e1b4e78bf45c214855f9a2e; no voters:
I20260811 18:59:05.163147 28807 raft_consensus.cc:2781] T 918921d81f954bdbb708d8bd832bd16f P b09217442e1b4e78bf45c214855f9a2e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:59:05.163247 28807 raft_consensus.cc:482] T 918921d81f954bdbb708d8bd832bd16f P b09217442e1b4e78bf45c214855f9a2e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:59:05.163290 28807 raft_consensus.cc:3037] T 918921d81f954bdbb708d8bd832bd16f P b09217442e1b4e78bf45c214855f9a2e [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:59:05.164309 28807 raft_consensus.cc:504] T 918921d81f954bdbb708d8bd832bd16f P b09217442e1b4e78bf45c214855f9a2e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07c141020dd24574b218d794f88c3fe1" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 41313 } } peers { permanent_uuid: "b09217442e1b4e78bf45c214855f9a2e" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 44951 } } peers { permanent_uuid: "160bce60bb1e4264a56aae68be947c3b" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 39085 } }
I20260811 18:59:05.164626 28807 leader_election.cc:288] T 918921d81f954bdbb708d8bd832bd16f P b09217442e1b4e78bf45c214855f9a2e [CANDIDATE]: Term 1 election: Requested vote from peers 07c141020dd24574b218d794f88c3fe1 (127.27.180.129:41313), 160bce60bb1e4264a56aae68be947c3b (127.27.180.130:39085)
I20260811 18:59:05.164929 28160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "918921d81f954bdbb708d8bd832bd16f" candidate_uuid: "b09217442e1b4e78bf45c214855f9a2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07c141020dd24574b218d794f88c3fe1"
I20260811 18:59:05.164999 28412 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "918921d81f954bdbb708d8bd832bd16f" candidate_uuid: "b09217442e1b4e78bf45c214855f9a2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "160bce60bb1e4264a56aae68be947c3b"
I20260811 18:59:05.165047 28160 raft_consensus.cc:3037] T 918921d81f954bdbb708d8bd832bd16f P 07c141020dd24574b218d794f88c3fe1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:59:05.165155 28412 raft_consensus.cc:3037] T 918921d81f954bdbb708d8bd832bd16f P 160bce60bb1e4264a56aae68be947c3b [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:59:05.166108 28160 raft_consensus.cc:2454] T 918921d81f954bdbb708d8bd832bd16f P 07c141020dd24574b218d794f88c3fe1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b09217442e1b4e78bf45c214855f9a2e in term 1.
I20260811 18:59:05.166231 28412 raft_consensus.cc:2454] T 918921d81f954bdbb708d8bd832bd16f P 160bce60bb1e4264a56aae68be947c3b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b09217442e1b4e78bf45c214855f9a2e in term 1.
I20260811 18:59:05.166409 28535 leader_election.cc:302] T 918921d81f954bdbb708d8bd832bd16f P b09217442e1b4e78bf45c214855f9a2e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07c141020dd24574b218d794f88c3fe1, b09217442e1b4e78bf45c214855f9a2e; no voters:
I20260811 18:59:05.166566 28807 raft_consensus.cc:2781] T 918921d81f954bdbb708d8bd832bd16f P b09217442e1b4e78bf45c214855f9a2e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:59:05.166778 28807 raft_consensus.cc:686] T 918921d81f954bdbb708d8bd832bd16f P b09217442e1b4e78bf45c214855f9a2e [term 1 LEADER]: Becoming Leader. State: Replica: b09217442e1b4e78bf45c214855f9a2e, State: Running, Role: LEADER
I20260811 18:59:05.167068 28807 consensus_queue.cc:237] T 918921d81f954bdbb708d8bd832bd16f P b09217442e1b4e78bf45c214855f9a2e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07c141020dd24574b218d794f88c3fe1" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 41313 } } peers { permanent_uuid: "b09217442e1b4e78bf45c214855f9a2e" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 44951 } } peers { permanent_uuid: "160bce60bb1e4264a56aae68be947c3b" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 39085 } }
I20260811 18:59:05.170185 27890 catalog_manager.cc:5495] T 918921d81f954bdbb708d8bd832bd16f P b09217442e1b4e78bf45c214855f9a2e reported cstate change: term changed from 0 to 1, leader changed from <none> to b09217442e1b4e78bf45c214855f9a2e (127.27.180.131). New cstate: current_term: 1 leader_uuid: "b09217442e1b4e78bf45c214855f9a2e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07c141020dd24574b218d794f88c3fe1" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 41313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b09217442e1b4e78bf45c214855f9a2e" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 44951 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "160bce60bb1e4264a56aae68be947c3b" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 39085 } health_report { overall_health: UNKNOWN } } }
I20260811 18:59:05.188066 28790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:05.184729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50320 (local address 127.27.180.131:44951)
0811 18:59:05.184839 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:05.184844 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:05.184870 (+ 26us) server_negotiation.cc:407] Connection header received
0811 18:59:05.184998 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:05.185000 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:05.185052 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:05.185161 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:59:05.185379 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:05.185894 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:05.186730 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:05.186950 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:05.187035 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:05.187318 (+ 283us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:59:05.187688 (+ 370us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:59:05.187793 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:05.187870 (+ 77us) server_negotiation.cc:299] Negotiation successful
0811 18:59:05.187924 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260811 18:59:05.193473 28160 raft_consensus.cc:1261] T 918921d81f954bdbb708d8bd832bd16f P 07c141020dd24574b218d794f88c3fe1 [term 1 FOLLOWER]: Refusing update from remote peer b09217442e1b4e78bf45c214855f9a2e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:59:05.193473 28412 raft_consensus.cc:1261] T 918921d81f954bdbb708d8bd832bd16f P 160bce60bb1e4264a56aae68be947c3b [term 1 FOLLOWER]: Refusing update from remote peer b09217442e1b4e78bf45c214855f9a2e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:59:05.194181 28810 consensus_queue.cc:1035] T 918921d81f954bdbb708d8bd832bd16f P b09217442e1b4e78bf45c214855f9a2e [LEADER]: Connected to new peer: Peer: permanent_uuid: "160bce60bb1e4264a56aae68be947c3b" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 39085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:59:05.194345 28807 consensus_queue.cc:1035] T 918921d81f954bdbb708d8bd832bd16f P b09217442e1b4e78bf45c214855f9a2e [LEADER]: Connected to new peer: Peer: permanent_uuid: "07c141020dd24574b218d794f88c3fe1" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 41313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:59:05.200942 28820 mvcc.cc:205] Tried to move back new op lower bound from 7317400556305551360 to 7317400556206481408. Current Snapshot: MvccSnapshot[applied={T|T < 7317400556305551360}]
I20260811 18:59:05.210005 27891 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41956:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:59:05.210160 27891 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41956:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:59:05.212476 27891 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6e52c276e0c64b8abfbdf33c1df0d141: Sending DeleteTablet for 3 replicas of tablet 918921d81f954bdbb708d8bd832bd16f
I20260811 18:59:05.213424 28612 tablet_service.cc:1514] Processing DeleteTablet for tablet 918921d81f954bdbb708d8bd832bd16f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:59:05 UTC) from {username='kudu', principal='kudu/127.27.180.190@KRBTEST.COM'} at 127.0.0.1:50312
I20260811 18:59:05.213438 28365 tablet_service.cc:1514] Processing DeleteTablet for tablet 918921d81f954bdbb708d8bd832bd16f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:59:05 UTC) from {username='kudu', principal='kudu/127.27.180.190@KRBTEST.COM'} at 127.0.0.1:37650
I20260811 18:59:05.213459 28123 tablet_service.cc:1514] Processing DeleteTablet for tablet 918921d81f954bdbb708d8bd832bd16f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:59:05 UTC) from {username='kudu', principal='kudu/127.27.180.190@KRBTEST.COM'} at 127.0.0.1:45204
I20260811 18:59:05.214133 28840 tablet_replica.cc:330] T 918921d81f954bdbb708d8bd832bd16f P b09217442e1b4e78bf45c214855f9a2e: stopping tablet replica
I20260811 18:59:05.214133 28843 tablet_replica.cc:330] T 918921d81f954bdbb708d8bd832bd16f P 07c141020dd24574b218d794f88c3fe1: stopping tablet replica
I20260811 18:59:05.214519 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 28014
I20260811 18:59:05.275542 28840 raft_consensus.cc:2229] T 918921d81f954bdbb708d8bd832bd16f P b09217442e1b4e78bf45c214855f9a2e [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:59:05.276153 28840 raft_consensus.cc:2258] T 918921d81f954bdbb708d8bd832bd16f P b09217442e1b4e78bf45c214855f9a2e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:59:05.278093 28840 ts_tablet_manager.cc:1905] T 918921d81f954bdbb708d8bd832bd16f P b09217442e1b4e78bf45c214855f9a2e: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 18:59:05.280972 27867 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.27.180.129:41313 (error 108)
I20260811 18:59:05.281060 28840 ts_tablet_manager.cc:1918] T 918921d81f954bdbb708d8bd832bd16f P b09217442e1b4e78bf45c214855f9a2e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:59:05.281404 28840 log.cc:1196] T 918921d81f954bdbb708d8bd832bd16f P b09217442e1b4e78bf45c214855f9a2e: Deleting WAL directory at /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786474496157106-28370-0/minicluster-data/ts-2/wal/wals/918921d81f954bdbb708d8bd832bd16f
I20260811 18:59:05.281804 28840 ts_tablet_manager.cc:1939] T 918921d81f954bdbb708d8bd832bd16f P b09217442e1b4e78bf45c214855f9a2e: Deleting consensus metadata
I20260811 18:59:05.281980 28842 tablet_replica.cc:330] T 918921d81f954bdbb708d8bd832bd16f P 160bce60bb1e4264a56aae68be947c3b: stopping tablet replica
I20260811 18:59:05.282110 28783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:05.281820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.129:41313 (local address 127.0.0.1:45206)
0811 18:59:05.281950 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:05.282020 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.129:41313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260811 18:59:05.282224 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 28240
W20260811 18:59:05.282305 27867 catalog_manager.cc:4584] TS 07c141020dd24574b218d794f88c3fe1 (127.27.180.129:41313): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 918921d81f954bdbb708d8bd832bd16f: Network error: Client connection negotiation failed: client connection to 127.27.180.129:41313: connect: Connection refused (error 111)
I20260811 18:59:05.282393 28842 raft_consensus.cc:2229] T 918921d81f954bdbb708d8bd832bd16f P 160bce60bb1e4264a56aae68be947c3b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:59:05.285594 27864 catalog_manager.cc:4841] TS b09217442e1b4e78bf45c214855f9a2e (127.27.180.131:44951): tablet 918921d81f954bdbb708d8bd832bd16f (table test-table [id=fa3a2b90463d45078660050e40989249]) successfully deleted
I20260811 18:59:05.292672 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 28507
I20260811 18:59:05.293355 28783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:05.292766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.130:39085 (local address 127.0.0.1:37654)
0811 18:59:05.293219 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:05.293276 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.130:39085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":373}
I20260811 18:59:05.303805 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 27836
2026-08-11T18:59:05Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2824 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-11T18:59:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:59:05Z Disabled control of system clock
I20260811 18:59:05.336313 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474496157106-28370-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474496157106-28370-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474496157106-28370-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.190:33171
--webserver_interface=127.27.180.190
--webserver_port=0
--builtin_ntp_servers=127.27.180.148:38045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.180.190:33171
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 18:59:05.443902 28849 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:59:05.444149 28849 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:59:05.444181 28849 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:59:05.448530 28849 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:59:05.448583 28849 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:59:05.448597 28849 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:59:05.448611 28849 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:59:05.448638 28849 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:59:05.453419 28849 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:38045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474496157106-28370-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474496157106-28370-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.180.190:33171
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.190:33171
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.180.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28849
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474496157106-28370-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:59:05.454445 28849 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:59:05.455775 28849 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:59:05.463860 28858 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:05.463915 28861 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:05.463852 28859 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:59:05.464246 28849 server_base.cc:1034] running on GCE node
I20260811 18:59:05.464788 28849 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:59:05.465893 28849 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:59:05.467084 28849 hybrid_clock.cc:648] HybridClock initialized: now 1786474745467069 us; error 25 us; skew 500 ppm
I20260811 18:59:05.468897 28849 webserver.cc:460] Webserver started at http://127.27.180.190:37993/ using document root <none> and password file <none>
I20260811 18:59:05.469478 28849 fs_manager.cc:359] Metadata directory not provided
I20260811 18:59:05.469553 28849 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:59:05.469764 28849 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:59:05.471802 28849 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474496157106-28370-0/minicluster-data/master-0/data/instance:
uuid: "2b6c3098118f4969820a694809b00d8a"
format_stamp: "Formatted at 2026-08-11 18:59:05 on dist-test-slave-2j7r"
server_key: "9130a3b4220247c0239bd3427ed82d65"
server_key_iv: "2a4e7449b84c4d4ce39699480151734f"
server_key_version: "encryptionkey@0"
I20260811 18:59:05.472363 28849 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474496157106-28370-0/minicluster-data/master-0/wal/instance:
uuid: "2b6c3098118f4969820a694809b00d8a"
format_stamp: "Formatted at 2026-08-11 18:59:05 on dist-test-slave-2j7r"
server_key: "9130a3b4220247c0239bd3427ed82d65"
server_key_iv: "2a4e7449b84c4d4ce39699480151734f"
server_key_version: "encryptionkey@0"
I20260811 18:59:05.475934 28849 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260811 18:59:05.478444 28873 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:05.479717 28849 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 18:59:05.479846 28849 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474496157106-28370-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474496157106-28370-0/minicluster-data/master-0/wal
uuid: "2b6c3098118f4969820a694809b00d8a"
format_stamp: "Formatted at 2026-08-11 18:59:05 on dist-test-slave-2j7r"
server_key: "9130a3b4220247c0239bd3427ed82d65"
server_key_iv: "2a4e7449b84c4d4ce39699480151734f"
server_key_version: "encryptionkey@0"
I20260811 18:59:05.479943 28849 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474496157106-28370-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474496157106-28370-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474496157106-28370-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:59:05.498164 28849 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:59:05.498819 28849 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:59:05.498982 28849 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:59:05.505762 28849 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.190:33171
I20260811 18:59:05.505777 28972 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.190:33171 every 8 connection(s)
I20260811 18:59:05.506659 28849 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
I20260811 18:59:05.509645 28973 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:59:05.511718 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 28849
I20260811 18:59:05.511837 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474496157106-28370-0/minicluster-data/master-0/wal/instance
I20260811 18:59:05.512108 28370 external_mini_cluster.cc:1428] Setting key bb1a899e08286dea09b1f96854f2074f
I20260811 18:59:05.515296 28973 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b6c3098118f4969820a694809b00d8a: Bootstrap starting.
I20260811 18:59:05.517431 28973 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2b6c3098118f4969820a694809b00d8a: Neither blocks nor log segments found. Creating new log.
I20260811 18:59:05.518131 28973 log.cc:824] T 00000000000000000000000000000000 P 2b6c3098118f4969820a694809b00d8a: Log is configured to *not* fsync() on all Append() calls
I20260811 18:59:05.519860 28973 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b6c3098118f4969820a694809b00d8a: No bootstrap required, opened a new log
I20260811 18:59:05.520838 28987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:05.513507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39338 (local address 127.27.180.190:33171)
0811 18:59:05.514139 (+ 632us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:05.514154 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:05.514200 (+ 46us) server_negotiation.cc:407] Connection header received
0811 18:59:05.514801 (+ 601us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:05.514823 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:05.515207 (+ 384us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:05.515500 (+ 293us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:05.515911 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:05.516879 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:05.517827 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:05.518315 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:05.518452 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:05.519261 (+ 809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:05.519266 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:05.519278 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:59:05.519650 (+ 372us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:05.519710 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:59:05.519730 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:05.519811 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:05.519940 (+ 129us) server_negotiation.cc:299] Negotiation successful
0811 18:59:05.520182 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292,"thread_start_us":133,"threads_started":1}
I20260811 18:59:05.523685 28973 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2b6c3098118f4969820a694809b00d8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b6c3098118f4969820a694809b00d8a" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 33171 } }
I20260811 18:59:05.523896 28973 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2b6c3098118f4969820a694809b00d8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:59:05.523953 28973 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2b6c3098118f4969820a694809b00d8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b6c3098118f4969820a694809b00d8a, State: Initialized, Role: FOLLOWER
I20260811 18:59:05.524356 28973 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2b6c3098118f4969820a694809b00d8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b6c3098118f4969820a694809b00d8a" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 33171 } }
I20260811 18:59:05.524488 28973 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2b6c3098118f4969820a694809b00d8a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:59:05.524551 28973 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2b6c3098118f4969820a694809b00d8a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:59:05.524646 28973 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2b6c3098118f4969820a694809b00d8a [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:59:05.525517 28973 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2b6c3098118f4969820a694809b00d8a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b6c3098118f4969820a694809b00d8a" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 33171 } }
I20260811 18:59:05.525821 28973 leader_election.cc:302] T 00000000000000000000000000000000 P 2b6c3098118f4969820a694809b00d8a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2b6c3098118f4969820a694809b00d8a; no voters:
I20260811 18:59:05.526129 28973 leader_election.cc:288] T 00000000000000000000000000000000 P 2b6c3098118f4969820a694809b00d8a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:59:05.526229 28990 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2b6c3098118f4969820a694809b00d8a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:59:05.526489 28990 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2b6c3098118f4969820a694809b00d8a [term 1 LEADER]: Becoming Leader. State: Replica: 2b6c3098118f4969820a694809b00d8a, State: Running, Role: LEADER
I20260811 18:59:05.526806 28990 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2b6c3098118f4969820a694809b00d8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b6c3098118f4969820a694809b00d8a" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 33171 } }
I20260811 18:59:05.527254 28973 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2b6c3098118f4969820a694809b00d8a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:59:05.528437 28994 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b6c3098118f4969820a694809b00d8a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2b6c3098118f4969820a694809b00d8a. Latest consensus state: current_term: 1 leader_uuid: "2b6c3098118f4969820a694809b00d8a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b6c3098118f4969820a694809b00d8a" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 33171 } } }
I20260811 18:59:05.528420 28992 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b6c3098118f4969820a694809b00d8a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2b6c3098118f4969820a694809b00d8a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b6c3098118f4969820a694809b00d8a" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 33171 } } }
I20260811 18:59:05.528570 28992 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b6c3098118f4969820a694809b00d8a [sys.catalog]: This master's current role is: LEADER
I20260811 18:59:05.528570 28994 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b6c3098118f4969820a694809b00d8a [sys.catalog]: This master's current role is: LEADER
I20260811 18:59:05.529031 29004 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:59:05.531875 29004 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:59:05.537637 29004 catalog_manager.cc:1347] Generated new cluster ID: 73a0d33373f44eeeb329ce6e9d845711
I20260811 18:59:05.537701 29004 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:59:05.547063 29004 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:59:05.547919 29004 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:59:05.552016 29004 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2b6c3098118f4969820a694809b00d8a: Generated new TSK 0
I20260811 18:59:05.552594 29004 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:59:05.562176 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474496157106-28370-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474496157106-28370-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474496157106-28370-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474496157106-28370-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.180.129:0
--local_ip_for_outbound_sockets=127.27.180.129
--webserver_interface=127.27.180.129
--webserver_port=0
--tserver_master_addrs=127.27.180.190:33171
--builtin_ntp_servers=127.27.180.148:38045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 18:59:05.672542 29017 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:59:05.672767 29017 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:59:05.672802 29017 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:59:05.676656 29017 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:59:05.676704 29017 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:59:05.676755 29017 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.180.129
I20260811 18:59:05.681352 29017 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:38045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474496157106-28370-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474496157106-28370-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.180.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474496157106-28370-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.180.129
--webserver_port=0
--tserver_master_addrs=127.27.180.190:33171
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29017
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.180.129
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474496157106-28370-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:59:05.682302 29017 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:59:05.683655 29017 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:59:05.693102 29025 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:05.693126 29026 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:59:05.693279 29017 server_base.cc:1034] running on GCE node
W20260811 18:59:05.693152 29033 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:59:05.693677 29017 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:59:05.694289 29017 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:59:05.695461 29017 hybrid_clock.cc:648] HybridClock initialized: now 1786474745695446 us; error 26 us; skew 500 ppm
I20260811 18:59:05.697130 29017 webserver.cc:460] Webserver started at http://127.27.180.129:42747/ using document root <none> and password file <none>
I20260811 18:59:05.697672 29017 fs_manager.cc:359] Metadata directory not provided
I20260811 18:59:05.697741 29017 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:59:05.697919 29017 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:59:05.699856 29017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474496157106-28370-0/minicluster-data/ts-0/data/instance:
uuid: "1ad96b424208409196572b962d2f05a6"
format_stamp: "Formatted at 2026-08-11 18:59:05 on dist-test-slave-2j7r"
server_key: "77ff08737144ab963b34be88a61e5a08"
server_key_iv: "0da08df9c7116427601537a537be78c2"
server_key_version: "encryptionkey@0"
I20260811 18:59:05.700374 29017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474496157106-28370-0/minicluster-data/ts-0/wal/instance:
uuid: "1ad96b424208409196572b962d2f05a6"
format_stamp: "Formatted at 2026-08-11 18:59:05 on dist-test-slave-2j7r"
server_key: "77ff08737144ab963b34be88a61e5a08"
server_key_iv: "0da08df9c7116427601537a537be78c2"
server_key_version: "encryptionkey@0"
I20260811 18:59:05.703750 29017 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260811 18:59:05.706074 29043 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:05.707195 29017 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 18:59:05.707317 29017 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474496157106-28370-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474496157106-28370-0/minicluster-data/ts-0/wal
uuid: "1ad96b424208409196572b962d2f05a6"
format_stamp: "Formatted at 2026-08-11 18:59:05 on dist-test-slave-2j7r"
server_key: "77ff08737144ab963b34be88a61e5a08"
server_key_iv: "0da08df9c7116427601537a537be78c2"
server_key_version: "encryptionkey@0"
I20260811 18:59:05.707408 29017 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474496157106-28370-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474496157106-28370-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474496157106-28370-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:59:05.721438 29017 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:59:05.722020 29017 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:59:05.722182 29017 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:59:05.722724 29017 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:59:05.723598 29017 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:59:05.723645 29017 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:05.723694 29017 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:59:05.723723 29017 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:05.732414 29017 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.129:45427
I20260811 18:59:05.732439 29254 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.129:45427 every 8 connection(s)
I20260811 18:59:05.733254 29017 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474496157106-28370-0/minicluster-data/ts-0/data/info.pb
I20260811 18:59:05.739023 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 29017
I20260811 18:59:05.739141 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474496157106-28370-0/minicluster-data/ts-0/wal/instance
I20260811 18:59:05.739430 28370 external_mini_cluster.cc:1428] Setting key 5dd522595b6e81bc111e94a28c347022
I20260811 18:59:05.741621 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474496157106-28370-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474496157106-28370-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474496157106-28370-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474496157106-28370-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.180.130:0
--local_ip_for_outbound_sockets=127.27.180.130
--webserver_interface=127.27.180.130
--webserver_port=0
--tserver_master_addrs=127.27.180.190:33171
--builtin_ntp_servers=127.27.180.148:38045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 18:59:05.742283 28987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:05.734904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.129:35723 (local address 127.27.180.190:33171)
0811 18:59:05.735021 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:05.735024 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:05.736020 (+ 996us) server_negotiation.cc:407] Connection header received
0811 18:59:05.737439 (+ 1419us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:05.737442 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:05.737502 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:05.737572 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:05.738420 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:05.739045 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:05.740337 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:05.740567 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:05.740657 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:05.741788 (+ 1131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:05.741792 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:05.741794 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:59:05.741810 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:05.741834 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:59:05.741846 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:05.741933 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:05.742059 (+ 126us) server_negotiation.cc:299] Negotiation successful
0811 18:59:05.742122 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 18:59:05.743016 29259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:05.735245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33171 (local address 127.27.180.129:35723)
0811 18:59:05.735871 (+ 626us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:05.735908 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 18:59:05.737233 (+ 1325us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:59:05.737746 (+ 513us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:59:05.737752 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:59:05.737840 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:59:05.738285 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:05.738298 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:05.739176 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:05.739181 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:05.740207 (+ 1026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:05.740218 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:05.740716 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:05.740721 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:05.740915 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:05.741526 (+ 611us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:59:05.741527 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:59:05.741567 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:59:05.741627 (+ 60us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:59:05.741934 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:59:05.741941 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:59:05.741945 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 18:59:05.742037 (+ 92us) client_negotiation.cc:240] Negotiation successful
0811 18:59:05.742250 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":114,"threads_started":1}
I20260811 18:59:05.744237 29255 heartbeater.cc:347] Connected to a master server at 127.27.180.190:33171
I20260811 18:59:05.744452 29255 heartbeater.cc:464] Registering TS with master...
I20260811 18:59:05.745013 29255 heartbeater.cc:511] Master 127.27.180.190:33171 requested a full tablet report, sending...
I20260811 18:59:05.746627 28905 ts_manager.cc:194] Registered new tserver with Master: 1ad96b424208409196572b962d2f05a6 (127.27.180.129:45427)
I20260811 18:59:05.747710 28905 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.27.180.129:35723
W20260811 18:59:05.852475 29262 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:59:05.852699 29262 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:59:05.852739 29262 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:59:05.856616 29262 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:59:05.856659 29262 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:59:05.856752 29262 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.180.130
I20260811 18:59:05.861449 29262 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:38045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474496157106-28370-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474496157106-28370-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.180.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474496157106-28370-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.180.130
--webserver_port=0
--tserver_master_addrs=127.27.180.190:33171
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29262
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.180.130
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474496157106-28370-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:59:05.862355 29262 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:59:05.863591 29262 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:59:05.872740 29281 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:05.872733 29279 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:05.872733 29288 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:59:05.872974 29262 server_base.cc:1034] running on GCE node
I20260811 18:59:05.873427 29262 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:59:05.873981 29262 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:59:05.875133 29262 hybrid_clock.cc:648] HybridClock initialized: now 1786474745875118 us; error 25 us; skew 500 ppm
I20260811 18:59:05.876814 29262 webserver.cc:460] Webserver started at http://127.27.180.130:44025/ using document root <none> and password file <none>
I20260811 18:59:05.877368 29262 fs_manager.cc:359] Metadata directory not provided
I20260811 18:59:05.877439 29262 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:59:05.877627 29262 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:59:05.879498 29262 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474496157106-28370-0/minicluster-data/ts-1/data/instance:
uuid: "602406459b0d4c3c9c69eadcbe1b5373"
format_stamp: "Formatted at 2026-08-11 18:59:05 on dist-test-slave-2j7r"
server_key: "db5f6cf170840fc62a6770a4f2396895"
server_key_iv: "6f21ba4846d4fca961c5a9f0d4c6d08f"
server_key_version: "encryptionkey@0"
I20260811 18:59:05.879999 29262 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474496157106-28370-0/minicluster-data/ts-1/wal/instance:
uuid: "602406459b0d4c3c9c69eadcbe1b5373"
format_stamp: "Formatted at 2026-08-11 18:59:05 on dist-test-slave-2j7r"
server_key: "db5f6cf170840fc62a6770a4f2396895"
server_key_iv: "6f21ba4846d4fca961c5a9f0d4c6d08f"
server_key_version: "encryptionkey@0"
I20260811 18:59:05.883401 29262 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260811 18:59:05.885696 29297 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:05.886806 29262 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 18:59:05.886904 29262 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474496157106-28370-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474496157106-28370-0/minicluster-data/ts-1/wal
uuid: "602406459b0d4c3c9c69eadcbe1b5373"
format_stamp: "Formatted at 2026-08-11 18:59:05 on dist-test-slave-2j7r"
server_key: "db5f6cf170840fc62a6770a4f2396895"
server_key_iv: "6f21ba4846d4fca961c5a9f0d4c6d08f"
server_key_version: "encryptionkey@0"
I20260811 18:59:05.886994 29262 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474496157106-28370-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474496157106-28370-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474496157106-28370-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:59:05.904115 29262 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:59:05.904729 29262 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:59:05.904886 29262 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:59:05.905458 29262 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:59:05.906340 29262 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:59:05.906394 29262 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:05.906447 29262 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:59:05.906474 29262 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:05.914911 29262 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.130:33667
I20260811 18:59:05.914938 29492 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.130:33667 every 8 connection(s)
I20260811 18:59:05.915745 29262 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474496157106-28370-0/minicluster-data/ts-1/data/info.pb
I20260811 18:59:05.918581 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 29262
I20260811 18:59:05.918711 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474496157106-28370-0/minicluster-data/ts-1/wal/instance
I20260811 18:59:05.918983 28370 external_mini_cluster.cc:1428] Setting key f17546db5aae25ec004d5a8ed81342bf
I20260811 18:59:05.920845 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474496157106-28370-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474496157106-28370-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474496157106-28370-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474496157106-28370-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.180.131:0
--local_ip_for_outbound_sockets=127.27.180.131
--webserver_interface=127.27.180.131
--webserver_port=0
--tserver_master_addrs=127.27.180.190:33171
--builtin_ntp_servers=127.27.180.148:38045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 18:59:05.924798 28987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:05.917464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.130:39039 (local address 127.27.180.190:33171)
0811 18:59:05.917599 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:05.917602 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:05.918598 (+ 996us) server_negotiation.cc:407] Connection header received
0811 18:59:05.920084 (+ 1486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:05.920087 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:05.920140 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:05.920211 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:05.921133 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:05.921658 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:05.922970 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:05.923195 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:05.923278 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:05.924381 (+ 1103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:05.924385 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:05.924387 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:59:05.924400 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:05.924421 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:59:05.924429 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:05.924481 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:05.924628 (+ 147us) server_negotiation.cc:299] Negotiation successful
0811 18:59:05.924684 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 18:59:05.925935 29496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:05.917815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33171 (local address 127.27.180.130:39039)
0811 18:59:05.918455 (+ 640us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:05.918494 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 18:59:05.919884 (+ 1390us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:59:05.920387 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:59:05.920394 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:59:05.920478 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:59:05.920962 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:05.920983 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:05.921782 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:05.921785 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:05.922835 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:05.922859 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:05.923338 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:05.923342 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:05.923530 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:05.924109 (+ 579us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:59:05.924111 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:59:05.924160 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:59:05.924235 (+ 75us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:59:05.924499 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:59:05.924508 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:59:05.924513 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 18:59:05.924629 (+ 116us) client_negotiation.cc:240] Negotiation successful
0811 18:59:05.924895 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":108,"threads_started":1}
I20260811 18:59:05.926980 29493 heartbeater.cc:347] Connected to a master server at 127.27.180.190:33171
I20260811 18:59:05.927194 29493 heartbeater.cc:464] Registering TS with master...
I20260811 18:59:05.927724 29493 heartbeater.cc:511] Master 127.27.180.190:33171 requested a full tablet report, sending...
I20260811 18:59:05.928692 28905 ts_manager.cc:194] Registered new tserver with Master: 602406459b0d4c3c9c69eadcbe1b5373 (127.27.180.130:33667)
I20260811 18:59:05.929289 28905 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.27.180.130:39039
W20260811 18:59:06.036064 29498 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:59:06.036303 29498 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:59:06.036340 29498 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:59:06.040275 29498 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:59:06.040323 29498 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:59:06.040373 29498 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.180.131
I20260811 18:59:06.044972 29498 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:38045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474496157106-28370-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474496157106-28370-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.180.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474496157106-28370-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.180.131
--webserver_port=0
--tserver_master_addrs=127.27.180.190:33171
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29498
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.180.131
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474496157106-28370-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:59:06.046044 29498 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:59:06.047369 29498 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:59:06.056706 29507 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:59:06.056810 29498 server_base.cc:1034] running on GCE node
W20260811 18:59:06.056720 29504 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:06.056728 29511 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:59:06.057399 29498 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:59:06.058024 29498 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:59:06.059186 29498 hybrid_clock.cc:648] HybridClock initialized: now 1786474746059169 us; error 30 us; skew 500 ppm
I20260811 18:59:06.061079 29498 webserver.cc:460] Webserver started at http://127.27.180.131:35797/ using document root <none> and password file <none>
I20260811 18:59:06.061772 29498 fs_manager.cc:359] Metadata directory not provided
I20260811 18:59:06.061858 29498 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:59:06.062078 29498 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:59:06.064157 29498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474496157106-28370-0/minicluster-data/ts-2/data/instance:
uuid: "176652cde2da41faacb638afee31891c"
format_stamp: "Formatted at 2026-08-11 18:59:06 on dist-test-slave-2j7r"
server_key: "7aaefa88038a3b10b92dac7358fbb06a"
server_key_iv: "4c5fabe19b36f8990f7072d07af32c99"
server_key_version: "encryptionkey@0"
I20260811 18:59:06.064718 29498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474496157106-28370-0/minicluster-data/ts-2/wal/instance:
uuid: "176652cde2da41faacb638afee31891c"
format_stamp: "Formatted at 2026-08-11 18:59:06 on dist-test-slave-2j7r"
server_key: "7aaefa88038a3b10b92dac7358fbb06a"
server_key_iv: "4c5fabe19b36f8990f7072d07af32c99"
server_key_version: "encryptionkey@0"
I20260811 18:59:06.068423 29498 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260811 18:59:06.070873 29520 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:06.072072 29498 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260811 18:59:06.072177 29498 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474496157106-28370-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474496157106-28370-0/minicluster-data/ts-2/wal
uuid: "176652cde2da41faacb638afee31891c"
format_stamp: "Formatted at 2026-08-11 18:59:06 on dist-test-slave-2j7r"
server_key: "7aaefa88038a3b10b92dac7358fbb06a"
server_key_iv: "4c5fabe19b36f8990f7072d07af32c99"
server_key_version: "encryptionkey@0"
I20260811 18:59:06.072299 29498 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474496157106-28370-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474496157106-28370-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474496157106-28370-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:59:06.089537 29498 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:59:06.090205 29498 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:59:06.090365 29498 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:59:06.090955 29498 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:59:06.091881 29498 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:59:06.091925 29498 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:06.091964 29498 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:59:06.091984 29498 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:06.099913 29498 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.131:37217
I20260811 18:59:06.099972 29736 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.131:37217 every 8 connection(s)
I20260811 18:59:06.100764 29498 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474496157106-28370-0/minicluster-data/ts-2/data/info.pb
I20260811 18:59:06.108517 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 29498
I20260811 18:59:06.108639 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474496157106-28370-0/minicluster-data/ts-2/wal/instance
I20260811 18:59:06.108974 28370 external_mini_cluster.cc:1428] Setting key 5084d0a229a0113a9307865972d19a40
I20260811 18:59:06.109447 28987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:06.102541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.131:39091 (local address 127.27.180.190:33171)
0811 18:59:06.102723 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:06.102728 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:06.103722 (+ 994us) server_negotiation.cc:407] Connection header received
0811 18:59:06.105075 (+ 1353us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:06.105077 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:06.105155 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:06.105239 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:06.106073 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:06.106578 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:06.107618 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:06.107843 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:06.107918 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:06.108945 (+ 1027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:06.108948 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:06.108950 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:59:06.108962 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:06.108981 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:59:06.108989 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:06.109054 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:06.109243 (+ 189us) server_negotiation.cc:299] Negotiation successful
0811 18:59:06.109304 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 18:59:06.110411 29741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:06.102941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33171 (local address 127.27.180.131:39091)
0811 18:59:06.103572 (+ 631us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:06.103611 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 18:59:06.104883 (+ 1272us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:59:06.105389 (+ 506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:59:06.105396 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:59:06.105476 (+ 80us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:59:06.105962 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:06.105974 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:06.106661 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:06.106663 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:06.107507 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:06.107516 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:06.107961 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:06.107964 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:06.108097 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:06.108658 (+ 561us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:59:06.108660 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:59:06.108712 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:59:06.108788 (+ 76us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:59:06.109069 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:59:06.109079 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:59:06.109108 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 18:59:06.109228 (+ 120us) client_negotiation.cc:240] Negotiation successful
0811 18:59:06.109463 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":118,"threads_started":1}
I20260811 18:59:06.111478 29737 heartbeater.cc:347] Connected to a master server at 127.27.180.190:33171
I20260811 18:59:06.111693 29737 heartbeater.cc:464] Registering TS with master...
I20260811 18:59:06.112151 29737 heartbeater.cc:511] Master 127.27.180.190:33171 requested a full tablet report, sending...
I20260811 18:59:06.113046 28904 ts_manager.cc:194] Registered new tserver with Master: 176652cde2da41faacb638afee31891c (127.27.180.131:37217)
I20260811 18:59:06.113621 28904 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.27.180.131:39091
I20260811 18:59:06.122787 28370 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 18:59:06.128649 28987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:06.124638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39352 (local address 127.27.180.190:33171)
0811 18:59:06.124805 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:06.124810 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:06.124980 (+ 170us) server_negotiation.cc:407] Connection header received
0811 18:59:06.125122 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:06.125125 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:06.125173 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:06.125257 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:06.125560 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:06.126107 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:06.127048 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:06.127280 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:06.127378 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:06.128223 (+ 845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:06.128227 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:06.128230 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:59:06.128245 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:06.128266 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:59:06.128275 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:06.128332 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:06.128453 (+ 121us) server_negotiation.cc:299] Negotiation successful
0811 18:59:06.128507 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 18:59:06.131321 28904 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39352:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:59:06.134291 28904 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:59:06.147742 29758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:06.142275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.130:33667 (local address 127.0.0.1:36644)
0811 18:59:06.142753 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:06.142799 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 18:59:06.142919 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:59:06.143716 (+ 797us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:59:06.143721 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:59:06.143743 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:59:06.143858 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:06.143865 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:06.145027 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:06.145030 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:06.146196 (+ 1166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:06.146206 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:06.147223 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:06.147226 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:06.147330 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:06.147366 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 18:59:06.147475 (+ 109us) client_negotiation.cc:240] Negotiation successful
0811 18:59:06.147598 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"mutex_wait_us":1,"thread_start_us":70,"threads_started":1}
I20260811 18:59:06.147758 29760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:06.142429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36644 (local address 127.27.180.130:33667)
0811 18:59:06.143223 (+ 794us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:06.143234 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:06.143259 (+ 25us) server_negotiation.cc:407] Connection header received
0811 18:59:06.143346 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:06.143351 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:06.143552 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:06.143707 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:59:06.143972 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:06.144929 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:06.146336 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:06.147130 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:06.147231 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:06.147410 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:06.147504 (+ 94us) server_negotiation.cc:299] Negotiation successful
0811 18:59:06.147600 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":670,"thread_start_us":134,"threads_started":1}
I20260811 18:59:06.148214 29761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:06.142926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.129:45427 (local address 127.0.0.1:44236)
0811 18:59:06.143792 (+ 866us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:06.143804 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 18:59:06.143909 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:59:06.144354 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:59:06.144356 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:59:06.144373 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:59:06.144458 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:06.144463 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:06.145853 (+ 1390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:06.145857 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:06.146724 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:06.146734 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:06.147821 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:06.147825 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:06.147931 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:06.147958 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 18:59:06.148018 (+ 60us) client_negotiation.cc:240] Negotiation successful
0811 18:59:06.148085 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":808,"mutex_wait_us":20,"thread_start_us":89,"threads_started":1}
I20260811 18:59:06.148736 29766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:06.142927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48718 (local address 127.27.180.131:37217)
0811 18:59:06.143647 (+ 720us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:06.143658 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:06.143682 (+ 24us) server_negotiation.cc:407] Connection header received
0811 18:59:06.143756 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:06.143761 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:06.143946 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:06.144085 (+ 139us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:59:06.144287 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:06.145418 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:06.147158 (+ 1740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:06.147880 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:06.148302 (+ 422us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:06.148475 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:06.148516 (+ 41us) server_negotiation.cc:299] Negotiation successful
0811 18:59:06.148583 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":620,"thread_start_us":100,"threads_started":1}
I20260811 18:59:06.148761 29768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:06.143329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44236 (local address 127.27.180.129:45427)
0811 18:59:06.143664 (+ 335us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:06.143674 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:06.143894 (+ 220us) server_negotiation.cc:407] Connection header received
0811 18:59:06.144058 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:06.144064 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:06.144230 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:06.144375 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:59:06.144555 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:06.145751 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:06.146838 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:06.147481 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:06.147594 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:06.147754 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:06.148477 (+ 723us) server_negotiation.cc:299] Negotiation successful
0811 18:59:06.148575 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":243,"thread_start_us":111,"threads_started":1}
I20260811 18:59:06.148399 29759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:06.142425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.131:37217 (local address 127.0.0.1:48718)
0811 18:59:06.143281 (+ 856us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:06.143295 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 18:59:06.143397 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:59:06.144069 (+ 672us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:59:06.144072 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:59:06.144091 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:59:06.144178 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:06.144184 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:06.145533 (+ 1349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:06.145537 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:06.147034 (+ 1497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:06.147045 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:06.148012 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:06.148016 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:06.148094 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:06.148113 (+ 19us) client_negotiation.cc:769] Sending connection context
0811 18:59:06.148158 (+ 45us) client_negotiation.cc:240] Negotiation successful
0811 18:59:06.148210 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":796,"mutex_wait_us":363,"thread_start_us":106,"threads_started":1}
I20260811 18:59:06.150789 29386 tablet_service.cc:1467] Processing CreateTablet for tablet 036d5365c06d482fa8e19cd4ff8aedfc (DEFAULT_TABLE table=test-table [id=223e9144c7754ea5931c3c3c63ff87bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:59:06.151386 29612 tablet_service.cc:1467] Processing CreateTablet for tablet 036d5365c06d482fa8e19cd4ff8aedfc (DEFAULT_TABLE table=test-table [id=223e9144c7754ea5931c3c3c63ff87bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:59:06.151708 29386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 036d5365c06d482fa8e19cd4ff8aedfc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:59:06.152194 29612 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 036d5365c06d482fa8e19cd4ff8aedfc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:59:06.155138 29136 tablet_service.cc:1467] Processing CreateTablet for tablet 036d5365c06d482fa8e19cd4ff8aedfc (DEFAULT_TABLE table=test-table [id=223e9144c7754ea5931c3c3c63ff87bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:59:06.156162 29136 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 036d5365c06d482fa8e19cd4ff8aedfc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:59:06.157794 29769 tablet_bootstrap.cc:492] T 036d5365c06d482fa8e19cd4ff8aedfc P 602406459b0d4c3c9c69eadcbe1b5373: Bootstrap starting.
I20260811 18:59:06.159453 29770 tablet_bootstrap.cc:492] T 036d5365c06d482fa8e19cd4ff8aedfc P 176652cde2da41faacb638afee31891c: Bootstrap starting.
I20260811 18:59:06.160552 29769 tablet_bootstrap.cc:654] T 036d5365c06d482fa8e19cd4ff8aedfc P 602406459b0d4c3c9c69eadcbe1b5373: Neither blocks nor log segments found. Creating new log.
I20260811 18:59:06.161458 29769 log.cc:824] T 036d5365c06d482fa8e19cd4ff8aedfc P 602406459b0d4c3c9c69eadcbe1b5373: Log is configured to *not* fsync() on all Append() calls
I20260811 18:59:06.161640 29770 tablet_bootstrap.cc:654] T 036d5365c06d482fa8e19cd4ff8aedfc P 176652cde2da41faacb638afee31891c: Neither blocks nor log segments found. Creating new log.
I20260811 18:59:06.162346 29770 log.cc:824] T 036d5365c06d482fa8e19cd4ff8aedfc P 176652cde2da41faacb638afee31891c: Log is configured to *not* fsync() on all Append() calls
I20260811 18:59:06.163085 29772 tablet_bootstrap.cc:492] T 036d5365c06d482fa8e19cd4ff8aedfc P 1ad96b424208409196572b962d2f05a6: Bootstrap starting.
I20260811 18:59:06.163615 29769 tablet_bootstrap.cc:492] T 036d5365c06d482fa8e19cd4ff8aedfc P 602406459b0d4c3c9c69eadcbe1b5373: No bootstrap required, opened a new log
I20260811 18:59:06.163877 29769 ts_tablet_manager.cc:1397] T 036d5365c06d482fa8e19cd4ff8aedfc P 602406459b0d4c3c9c69eadcbe1b5373: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 18:59:06.164259 29770 tablet_bootstrap.cc:492] T 036d5365c06d482fa8e19cd4ff8aedfc P 176652cde2da41faacb638afee31891c: No bootstrap required, opened a new log
I20260811 18:59:06.164450 29770 ts_tablet_manager.cc:1397] T 036d5365c06d482fa8e19cd4ff8aedfc P 176652cde2da41faacb638afee31891c: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260811 18:59:06.165210 29772 tablet_bootstrap.cc:654] T 036d5365c06d482fa8e19cd4ff8aedfc P 1ad96b424208409196572b962d2f05a6: Neither blocks nor log segments found. Creating new log.
I20260811 18:59:06.166015 29772 log.cc:824] T 036d5365c06d482fa8e19cd4ff8aedfc P 1ad96b424208409196572b962d2f05a6: Log is configured to *not* fsync() on all Append() calls
I20260811 18:59:06.167477 29769 raft_consensus.cc:348] T 036d5365c06d482fa8e19cd4ff8aedfc P 602406459b0d4c3c9c69eadcbe1b5373 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602406459b0d4c3c9c69eadcbe1b5373" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 33667 } } peers { permanent_uuid: "176652cde2da41faacb638afee31891c" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 37217 } } peers { permanent_uuid: "1ad96b424208409196572b962d2f05a6" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 45427 } }
I20260811 18:59:06.167694 29769 raft_consensus.cc:374] T 036d5365c06d482fa8e19cd4ff8aedfc P 602406459b0d4c3c9c69eadcbe1b5373 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:59:06.167765 29769 raft_consensus.cc:729] T 036d5365c06d482fa8e19cd4ff8aedfc P 602406459b0d4c3c9c69eadcbe1b5373 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 602406459b0d4c3c9c69eadcbe1b5373, State: Initialized, Role: FOLLOWER
I20260811 18:59:06.167953 29772 tablet_bootstrap.cc:492] T 036d5365c06d482fa8e19cd4ff8aedfc P 1ad96b424208409196572b962d2f05a6: No bootstrap required, opened a new log
I20260811 18:59:06.168179 29772 ts_tablet_manager.cc:1397] T 036d5365c06d482fa8e19cd4ff8aedfc P 1ad96b424208409196572b962d2f05a6: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.005s
I20260811 18:59:06.168181 29769 consensus_queue.cc:260] T 036d5365c06d482fa8e19cd4ff8aedfc P 602406459b0d4c3c9c69eadcbe1b5373 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602406459b0d4c3c9c69eadcbe1b5373" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 33667 } } peers { permanent_uuid: "176652cde2da41faacb638afee31891c" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 37217 } } peers { permanent_uuid: "1ad96b424208409196572b962d2f05a6" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 45427 } }
I20260811 18:59:06.168524 29770 raft_consensus.cc:348] T 036d5365c06d482fa8e19cd4ff8aedfc P 176652cde2da41faacb638afee31891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602406459b0d4c3c9c69eadcbe1b5373" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 33667 } } peers { permanent_uuid: "176652cde2da41faacb638afee31891c" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 37217 } } peers { permanent_uuid: "1ad96b424208409196572b962d2f05a6" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 45427 } }
I20260811 18:59:06.168783 29770 raft_consensus.cc:374] T 036d5365c06d482fa8e19cd4ff8aedfc P 176652cde2da41faacb638afee31891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:59:06.168825 29493 heartbeater.cc:503] Master 127.27.180.190:33171 was elected leader, sending a full tablet report...
I20260811 18:59:06.168843 29770 raft_consensus.cc:729] T 036d5365c06d482fa8e19cd4ff8aedfc P 176652cde2da41faacb638afee31891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 176652cde2da41faacb638afee31891c, State: Initialized, Role: FOLLOWER
I20260811 18:59:06.169286 29770 consensus_queue.cc:260] T 036d5365c06d482fa8e19cd4ff8aedfc P 176652cde2da41faacb638afee31891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602406459b0d4c3c9c69eadcbe1b5373" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 33667 } } peers { permanent_uuid: "176652cde2da41faacb638afee31891c" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 37217 } } peers { permanent_uuid: "1ad96b424208409196572b962d2f05a6" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 45427 } }
I20260811 18:59:06.170064 29737 heartbeater.cc:503] Master 127.27.180.190:33171 was elected leader, sending a full tablet report...
I20260811 18:59:06.170250 29770 ts_tablet_manager.cc:1428] T 036d5365c06d482fa8e19cd4ff8aedfc P 176652cde2da41faacb638afee31891c: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 18:59:06.171372 29769 ts_tablet_manager.cc:1428] T 036d5365c06d482fa8e19cd4ff8aedfc P 602406459b0d4c3c9c69eadcbe1b5373: Time spent starting tablet: real 0.007s user 0.000s sys 0.005s
I20260811 18:59:06.172092 29772 raft_consensus.cc:348] T 036d5365c06d482fa8e19cd4ff8aedfc P 1ad96b424208409196572b962d2f05a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602406459b0d4c3c9c69eadcbe1b5373" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 33667 } } peers { permanent_uuid: "176652cde2da41faacb638afee31891c" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 37217 } } peers { permanent_uuid: "1ad96b424208409196572b962d2f05a6" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 45427 } }
I20260811 18:59:06.172325 29772 raft_consensus.cc:374] T 036d5365c06d482fa8e19cd4ff8aedfc P 1ad96b424208409196572b962d2f05a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:59:06.172415 29772 raft_consensus.cc:729] T 036d5365c06d482fa8e19cd4ff8aedfc P 1ad96b424208409196572b962d2f05a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ad96b424208409196572b962d2f05a6, State: Initialized, Role: FOLLOWER
I20260811 18:59:06.172820 29772 consensus_queue.cc:260] T 036d5365c06d482fa8e19cd4ff8aedfc P 1ad96b424208409196572b962d2f05a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602406459b0d4c3c9c69eadcbe1b5373" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 33667 } } peers { permanent_uuid: "176652cde2da41faacb638afee31891c" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 37217 } } peers { permanent_uuid: "1ad96b424208409196572b962d2f05a6" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 45427 } }
I20260811 18:59:06.173796 29772 ts_tablet_manager.cc:1428] T 036d5365c06d482fa8e19cd4ff8aedfc P 1ad96b424208409196572b962d2f05a6: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260811 18:59:06.173844 29255 heartbeater.cc:503] Master 127.27.180.190:33171 was elected leader, sending a full tablet report...
I20260811 18:59:06.212855 29783 raft_consensus.cc:482] T 036d5365c06d482fa8e19cd4ff8aedfc P 176652cde2da41faacb638afee31891c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:59:06.213100 29783 raft_consensus.cc:504] T 036d5365c06d482fa8e19cd4ff8aedfc P 176652cde2da41faacb638afee31891c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602406459b0d4c3c9c69eadcbe1b5373" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 33667 } } peers { permanent_uuid: "176652cde2da41faacb638afee31891c" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 37217 } } peers { permanent_uuid: "1ad96b424208409196572b962d2f05a6" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 45427 } }
I20260811 18:59:06.214248 29783 leader_election.cc:288] T 036d5365c06d482fa8e19cd4ff8aedfc P 176652cde2da41faacb638afee31891c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 602406459b0d4c3c9c69eadcbe1b5373 (127.27.180.130:33667), 1ad96b424208409196572b962d2f05a6 (127.27.180.129:45427)
I20260811 18:59:06.218182 29741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:06.214503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.130:33667 (local address 127.27.180.131:43785)
0811 18:59:06.214615 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:06.214631 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 18:59:06.214725 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:59:06.215087 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:59:06.215090 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:59:06.215108 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:59:06.215211 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:06.215216 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:06.216095 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:06.216098 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:06.217011 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:06.217021 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:06.217764 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:06.217768 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:06.217874 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:06.217902 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 18:59:06.217981 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 18:59:06.218056 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260811 18:59:06.218183 29760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:06.214611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.131:43785 (local address 127.27.180.130:33667)
0811 18:59:06.214738 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:06.214743 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:06.214797 (+ 54us) server_negotiation.cc:407] Connection header received
0811 18:59:06.214880 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:06.214884 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:06.214937 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:06.215009 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:59:06.215328 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:06.215984 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:06.217151 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:06.217664 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:06.217756 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:06.217797 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:06.217991 (+ 194us) server_negotiation.cc:299] Negotiation successful
0811 18:59:06.218048 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 18:59:06.218537 29768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:06.214904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.131:46819 (local address 127.27.180.129:45427)
0811 18:59:06.215025 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:06.215029 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:06.215294 (+ 265us) server_negotiation.cc:407] Connection header received
0811 18:59:06.215415 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:06.215417 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:06.215460 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:06.215515 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:59:06.215795 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:06.216398 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:06.217511 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:06.217978 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:06.218079 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:06.218124 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:06.218347 (+ 223us) server_negotiation.cc:299] Negotiation successful
0811 18:59:06.218426 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 18:59:06.218873 29786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:06.214771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.129:45427 (local address 127.27.180.131:46819)
0811 18:59:06.215190 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:06.215203 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:59:06.215327 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:59:06.215545 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:59:06.215548 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:59:06.215566 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:59:06.215691 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:06.215696 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:06.216514 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:06.216517 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:06.217398 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:06.217409 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:06.218099 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:06.218103 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:06.218193 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:06.218217 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 18:59:06.218673 (+ 456us) client_negotiation.cc:240] Negotiation successful
0811 18:59:06.218756 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":137,"threads_started":1}
I20260811 18:59:06.218885 29415 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "036d5365c06d482fa8e19cd4ff8aedfc" candidate_uuid: "176652cde2da41faacb638afee31891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "602406459b0d4c3c9c69eadcbe1b5373" is_pre_election: true
I20260811 18:59:06.219228 29415 raft_consensus.cc:2454] T 036d5365c06d482fa8e19cd4ff8aedfc P 602406459b0d4c3c9c69eadcbe1b5373 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 176652cde2da41faacb638afee31891c in term 0.
I20260811 18:59:06.219568 29173 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "036d5365c06d482fa8e19cd4ff8aedfc" candidate_uuid: "176652cde2da41faacb638afee31891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ad96b424208409196572b962d2f05a6" is_pre_election: true
I20260811 18:59:06.219722 29522 leader_election.cc:302] T 036d5365c06d482fa8e19cd4ff8aedfc P 176652cde2da41faacb638afee31891c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 176652cde2da41faacb638afee31891c, 602406459b0d4c3c9c69eadcbe1b5373; no voters:
I20260811 18:59:06.219882 29173 raft_consensus.cc:2454] T 036d5365c06d482fa8e19cd4ff8aedfc P 1ad96b424208409196572b962d2f05a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 176652cde2da41faacb638afee31891c in term 0.
I20260811 18:59:06.220014 29783 raft_consensus.cc:2781] T 036d5365c06d482fa8e19cd4ff8aedfc P 176652cde2da41faacb638afee31891c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:59:06.220108 29783 raft_consensus.cc:482] T 036d5365c06d482fa8e19cd4ff8aedfc P 176652cde2da41faacb638afee31891c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:59:06.220161 29783 raft_consensus.cc:3037] T 036d5365c06d482fa8e19cd4ff8aedfc P 176652cde2da41faacb638afee31891c [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:59:06.221253 29783 raft_consensus.cc:504] T 036d5365c06d482fa8e19cd4ff8aedfc P 176652cde2da41faacb638afee31891c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602406459b0d4c3c9c69eadcbe1b5373" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 33667 } } peers { permanent_uuid: "176652cde2da41faacb638afee31891c" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 37217 } } peers { permanent_uuid: "1ad96b424208409196572b962d2f05a6" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 45427 } }
I20260811 18:59:06.221566 29783 leader_election.cc:288] T 036d5365c06d482fa8e19cd4ff8aedfc P 176652cde2da41faacb638afee31891c [CANDIDATE]: Term 1 election: Requested vote from peers 602406459b0d4c3c9c69eadcbe1b5373 (127.27.180.130:33667), 1ad96b424208409196572b962d2f05a6 (127.27.180.129:45427)
I20260811 18:59:06.221925 29173 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "036d5365c06d482fa8e19cd4ff8aedfc" candidate_uuid: "176652cde2da41faacb638afee31891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ad96b424208409196572b962d2f05a6"
I20260811 18:59:06.221941 29415 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "036d5365c06d482fa8e19cd4ff8aedfc" candidate_uuid: "176652cde2da41faacb638afee31891c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "602406459b0d4c3c9c69eadcbe1b5373"
I20260811 18:59:06.222033 29173 raft_consensus.cc:3037] T 036d5365c06d482fa8e19cd4ff8aedfc P 1ad96b424208409196572b962d2f05a6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:59:06.222047 29415 raft_consensus.cc:3037] T 036d5365c06d482fa8e19cd4ff8aedfc P 602406459b0d4c3c9c69eadcbe1b5373 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:59:06.223042 29173 raft_consensus.cc:2454] T 036d5365c06d482fa8e19cd4ff8aedfc P 1ad96b424208409196572b962d2f05a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 176652cde2da41faacb638afee31891c in term 1.
I20260811 18:59:06.223076 29415 raft_consensus.cc:2454] T 036d5365c06d482fa8e19cd4ff8aedfc P 602406459b0d4c3c9c69eadcbe1b5373 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 176652cde2da41faacb638afee31891c in term 1.
I20260811 18:59:06.223376 29522 leader_election.cc:302] T 036d5365c06d482fa8e19cd4ff8aedfc P 176652cde2da41faacb638afee31891c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 176652cde2da41faacb638afee31891c, 602406459b0d4c3c9c69eadcbe1b5373; no voters:
I20260811 18:59:06.223536 29783 raft_consensus.cc:2781] T 036d5365c06d482fa8e19cd4ff8aedfc P 176652cde2da41faacb638afee31891c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:59:06.223784 29783 raft_consensus.cc:686] T 036d5365c06d482fa8e19cd4ff8aedfc P 176652cde2da41faacb638afee31891c [term 1 LEADER]: Becoming Leader. State: Replica: 176652cde2da41faacb638afee31891c, State: Running, Role: LEADER
I20260811 18:59:06.224082 29783 consensus_queue.cc:237] T 036d5365c06d482fa8e19cd4ff8aedfc P 176652cde2da41faacb638afee31891c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602406459b0d4c3c9c69eadcbe1b5373" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 33667 } } peers { permanent_uuid: "176652cde2da41faacb638afee31891c" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 37217 } } peers { permanent_uuid: "1ad96b424208409196572b962d2f05a6" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 45427 } }
I20260811 18:59:06.227401 28904 catalog_manager.cc:5495] T 036d5365c06d482fa8e19cd4ff8aedfc P 176652cde2da41faacb638afee31891c reported cstate change: term changed from 0 to 1, leader changed from <none> to 176652cde2da41faacb638afee31891c (127.27.180.131). New cstate: current_term: 1 leader_uuid: "176652cde2da41faacb638afee31891c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "602406459b0d4c3c9c69eadcbe1b5373" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 33667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "176652cde2da41faacb638afee31891c" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 37217 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ad96b424208409196572b962d2f05a6" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 45427 } health_report { overall_health: UNKNOWN } } }
W20260811 18:59:06.234352 29258 tablet.cc:2369] T 036d5365c06d482fa8e19cd4ff8aedfc P 1ad96b424208409196572b962d2f05a6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:59:06.246418 29766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:06.243095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48732 (local address 127.27.180.131:37217)
0811 18:59:06.243200 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:06.243204 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:06.243215 (+ 11us) server_negotiation.cc:407] Connection header received
0811 18:59:06.243357 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:06.243359 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:06.243411 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:06.243497 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:59:06.243776 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:06.244279 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:06.245066 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:06.245328 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:06.245404 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:06.245701 (+ 297us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:59:06.246112 (+ 411us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:59:06.246200 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:06.246243 (+ 43us) server_negotiation.cc:299] Negotiation successful
0811 18:59:06.246289 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260811 18:59:06.251969 29173 raft_consensus.cc:1261] T 036d5365c06d482fa8e19cd4ff8aedfc P 1ad96b424208409196572b962d2f05a6 [term 1 FOLLOWER]: Refusing update from remote peer 176652cde2da41faacb638afee31891c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:59:06.252012 29415 raft_consensus.cc:1261] T 036d5365c06d482fa8e19cd4ff8aedfc P 602406459b0d4c3c9c69eadcbe1b5373 [term 1 FOLLOWER]: Refusing update from remote peer 176652cde2da41faacb638afee31891c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:59:06.252591 29783 consensus_queue.cc:1035] T 036d5365c06d482fa8e19cd4ff8aedfc P 176652cde2da41faacb638afee31891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ad96b424208409196572b962d2f05a6" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 45427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:59:06.252775 29788 consensus_queue.cc:1035] T 036d5365c06d482fa8e19cd4ff8aedfc P 176652cde2da41faacb638afee31891c [LEADER]: Connected to new peer: Peer: permanent_uuid: "602406459b0d4c3c9c69eadcbe1b5373" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 33667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:59:06.260164 29793 mvcc.cc:205] Tried to move back new op lower bound from 7317400560641409024 to 7317400560536518656. Current Snapshot: MvccSnapshot[applied={T|T < 7317400560641409024}]
I20260811 18:59:06.261358 29794 mvcc.cc:205] Tried to move back new op lower bound from 7317400560641409024 to 7317400560536518656. Current Snapshot: MvccSnapshot[applied={T|T < 7317400560641409024}]
I20260811 18:59:06.269356 28905 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39352:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:59:06.269522 28905 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39352:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:59:06.271806 28905 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2b6c3098118f4969820a694809b00d8a: Sending DeleteTablet for 3 replicas of tablet 036d5365c06d482fa8e19cd4ff8aedfc
I20260811 18:59:06.272616 29612 tablet_service.cc:1514] Processing DeleteTablet for tablet 036d5365c06d482fa8e19cd4ff8aedfc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:59:06 UTC) from {username='slave'} at 127.0.0.1:48718
I20260811 18:59:06.272665 29136 tablet_service.cc:1514] Processing DeleteTablet for tablet 036d5365c06d482fa8e19cd4ff8aedfc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:59:06 UTC) from {username='slave'} at 127.0.0.1:44236
I20260811 18:59:06.273079 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 29017
I20260811 18:59:06.273138 29386 tablet_service.cc:1514] Processing DeleteTablet for tablet 036d5365c06d482fa8e19cd4ff8aedfc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:59:06 UTC) from {username='slave'} at 127.0.0.1:36644
I20260811 18:59:06.279207 29815 tablet_replica.cc:330] T 036d5365c06d482fa8e19cd4ff8aedfc P 602406459b0d4c3c9c69eadcbe1b5373: stopping tablet replica
I20260811 18:59:06.273183 29813 tablet_replica.cc:330] T 036d5365c06d482fa8e19cd4ff8aedfc P 176652cde2da41faacb638afee31891c: stopping tablet replica
I20260811 18:59:06.279734 29815 raft_consensus.cc:2229] T 036d5365c06d482fa8e19cd4ff8aedfc P 602406459b0d4c3c9c69eadcbe1b5373 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:59:06.279982 29813 raft_consensus.cc:2229] T 036d5365c06d482fa8e19cd4ff8aedfc P 176652cde2da41faacb638afee31891c [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:59:06.280268 29815 raft_consensus.cc:2258] T 036d5365c06d482fa8e19cd4ff8aedfc P 602406459b0d4c3c9c69eadcbe1b5373 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:59:06.280447 29813 raft_consensus.cc:2258] T 036d5365c06d482fa8e19cd4ff8aedfc P 176652cde2da41faacb638afee31891c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:59:06.284173 29815 ts_tablet_manager.cc:1905] T 036d5365c06d482fa8e19cd4ff8aedfc P 602406459b0d4c3c9c69eadcbe1b5373: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:59:06.287035 29815 ts_tablet_manager.cc:1918] T 036d5365c06d482fa8e19cd4ff8aedfc P 602406459b0d4c3c9c69eadcbe1b5373: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:59:06.287138 29815 log.cc:1196] T 036d5365c06d482fa8e19cd4ff8aedfc P 602406459b0d4c3c9c69eadcbe1b5373: Deleting WAL directory at /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474496157106-28370-0/minicluster-data/ts-1/wal/wals/036d5365c06d482fa8e19cd4ff8aedfc
I20260811 18:59:06.287452 29815 ts_tablet_manager.cc:1939] T 036d5365c06d482fa8e19cd4ff8aedfc P 602406459b0d4c3c9c69eadcbe1b5373: Deleting consensus metadata
I20260811 18:59:06.289799 28876 catalog_manager.cc:4841] TS 602406459b0d4c3c9c69eadcbe1b5373 (127.27.180.130:33667): tablet 036d5365c06d482fa8e19cd4ff8aedfc (table test-table [id=223e9144c7754ea5931c3c3c63ff87bb]) successfully deleted
I20260811 18:59:06.290756 29813 ts_tablet_manager.cc:1905] T 036d5365c06d482fa8e19cd4ff8aedfc P 176652cde2da41faacb638afee31891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:59:06.293401 29813 ts_tablet_manager.cc:1918] T 036d5365c06d482fa8e19cd4ff8aedfc P 176652cde2da41faacb638afee31891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:59:06.293503 29813 log.cc:1196] T 036d5365c06d482fa8e19cd4ff8aedfc P 176652cde2da41faacb638afee31891c: Deleting WAL directory at /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786474496157106-28370-0/minicluster-data/ts-2/wal/wals/036d5365c06d482fa8e19cd4ff8aedfc
I20260811 18:59:06.293810 29813 ts_tablet_manager.cc:1939] T 036d5365c06d482fa8e19cd4ff8aedfc P 176652cde2da41faacb638afee31891c: Deleting consensus metadata
I20260811 18:59:06.295017 28874 catalog_manager.cc:4841] TS 176652cde2da41faacb638afee31891c (127.27.180.131:37217): tablet 036d5365c06d482fa8e19cd4ff8aedfc (table test-table [id=223e9144c7754ea5931c3c3c63ff87bb]) successfully deleted
W20260811 18:59:06.298141 28876 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.27.180.129:45427 (error 108)
I20260811 18:59:06.298377 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 29262
I20260811 18:59:06.301710 29761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:06.301395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.129:45427 (local address 127.0.0.1:44240)
0811 18:59:06.301560 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:06.301629 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.129:45427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
W20260811 18:59:06.301869 28876 catalog_manager.cc:4584] TS 1ad96b424208409196572b962d2f05a6 (127.27.180.129:45427): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 036d5365c06d482fa8e19cd4ff8aedfc: Network error: Client connection negotiation failed: client connection to 127.27.180.129:45427: connect: Connection refused (error 111)
I20260811 18:59:06.310051 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 29498
I20260811 18:59:06.331679 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 28849
2026-08-11T18:59:06Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1028 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474496157106-28370-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:59:06 dist-test-slave-2j7r krb5kdc[29821](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:59:06 dist-test-slave-2j7r krb5kdc[29821](info): set up 2 sockets
Aug 11 18:59:06 dist-test-slave-2j7r krb5kdc[29821](info): commencing operation
krb5kdc: starting...
W20260811 18:59:07.527019 28370 mini_kdc.cc:121] Time spent starting KDC: real 1.170s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474496157106-28370-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474496157106-28370-0/krb5kdc/test-user.keytab.
Aug 11 18:59:07 dist-test-slave-2j7r krb5kdc[29821](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474747, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:59:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:59:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.180.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.180.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.180.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab.
Entry for principal kudu/127.27.180.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab.
I20260811 18:59:07.648240 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474496157106-28370-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474496157106-28370-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474496157106-28370-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.190:42205
--webserver_interface=127.27.180.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab
--principal=kudu/127.27.180.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:34457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.180.190:42205
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:59:07.760422 29844 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:59:07.760706 29844 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:59:07.760764 29844 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:59:07.764904 29844 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:59:07.764972 29844 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:59:07.765000 29844 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:59:07.765021 29844 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:59:07.765048 29844 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:59:07.770447 29844 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:34457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474496157106-28370-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474496157106-28370-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.180.190:42205
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab
--principal=kudu/127.27.180.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.190:42205
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.180.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29844
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474496157106-28370-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:59:07.771483 29844 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:59:07.772496 29844 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:59:07.780804 29854 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:59:07.780942 29844 server_base.cc:1034] running on GCE node
W20260811 18:59:07.780813 29860 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:07.780818 29856 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:59:07.781674 29844 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:59:07.782753 29844 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:59:07.783919 29844 hybrid_clock.cc:648] HybridClock initialized: now 1786474747783900 us; error 29 us; skew 500 ppm
Aug 11 18:59:07 dist-test-slave-2j7r krb5kdc[29821](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474747, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:59:07.803136 29844 init.cc:377] Logged in from keytab as kudu/127.27.180.190@KRBTEST.COM (short username kudu)
I20260811 18:59:07.804138 29844 webserver.cc:460] Webserver started at http://127.27.180.190:39641/ using document root <none> and password file <none>
I20260811 18:59:07.804698 29844 fs_manager.cc:359] Metadata directory not provided
I20260811 18:59:07.804767 29844 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:59:07.804984 29844 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:59:07.807118 29844 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474496157106-28370-0/minicluster-data/master-0/data/instance:
uuid: "585cdb722eb84450938396f4093b53bb"
format_stamp: "Formatted at 2026-08-11 18:59:07 on dist-test-slave-2j7r"
server_key: "1a9688c0d586119f7b0d99ca849f6fd8"
server_key_iv: "237824caab9f15169ed9c2128a1ea58f"
server_key_version: "encryptionkey@0"
I20260811 18:59:07.807711 29844 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474496157106-28370-0/minicluster-data/master-0/wal/instance:
uuid: "585cdb722eb84450938396f4093b53bb"
format_stamp: "Formatted at 2026-08-11 18:59:07 on dist-test-slave-2j7r"
server_key: "1a9688c0d586119f7b0d99ca849f6fd8"
server_key_iv: "237824caab9f15169ed9c2128a1ea58f"
server_key_version: "encryptionkey@0"
I20260811 18:59:07.811494 29844 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 18:59:07.813877 29872 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:07.815004 29844 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 18:59:07.815122 29844 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474496157106-28370-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474496157106-28370-0/minicluster-data/master-0/wal
uuid: "585cdb722eb84450938396f4093b53bb"
format_stamp: "Formatted at 2026-08-11 18:59:07 on dist-test-slave-2j7r"
server_key: "1a9688c0d586119f7b0d99ca849f6fd8"
server_key_iv: "237824caab9f15169ed9c2128a1ea58f"
server_key_version: "encryptionkey@0"
I20260811 18:59:07.815263 29844 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474496157106-28370-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474496157106-28370-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474496157106-28370-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:59:07.833016 29844 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:59:07.835724 29844 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:59:07.835886 29844 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:59:07.842608 29844 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.190:42205
I20260811 18:59:07.842619 29977 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.190:42205 every 8 connection(s)
I20260811 18:59:07.843503 29844 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
I20260811 18:59:07.845214 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 29844
I20260811 18:59:07.845363 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474496157106-28370-0/minicluster-data/master-0/wal/instance
I20260811 18:59:07.845631 28370 external_mini_cluster.cc:1428] Setting key 30bca2eaffac3bb55127b3e0aeb545f2
I20260811 18:59:07.846298 29978 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:59:07.852480 29978 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 585cdb722eb84450938396f4093b53bb: Bootstrap starting.
Aug 11 18:59:07 dist-test-slave-2j7r krb5kdc[29821](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474747, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM
I20260811 18:59:07.855026 29978 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 585cdb722eb84450938396f4093b53bb: Neither blocks nor log segments found. Creating new log.
I20260811 18:59:07.855883 29978 log.cc:824] T 00000000000000000000000000000000 P 585cdb722eb84450938396f4093b53bb: Log is configured to *not* fsync() on all Append() calls
I20260811 18:59:07.858090 29978 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 585cdb722eb84450938396f4093b53bb: No bootstrap required, opened a new log
I20260811 18:59:07.859414 29982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:07.846855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42458 (local address 127.27.180.190:42205)
0811 18:59:07.847304 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:07.847321 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:07.847361 (+ 40us) server_negotiation.cc:407] Connection header received
0811 18:59:07.848012 (+ 651us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:07.848028 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:07.848359 (+ 331us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:07.848684 (+ 325us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:07.849576 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:07.850576 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:07.851499 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:07.851987 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:07.852152 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:07.854899 (+ 2747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:07.854904 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:07.854917 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:59:07.854946 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:07.856747 (+ 1801us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:07.857241 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:07.857246 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:07.857251 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:07.857364 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:07.857647 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:07.857649 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:07.857650 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:07.858014 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:07.858151 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:07.858455 (+ 304us) server_negotiation.cc:299] Negotiation successful
0811 18:59:07.858723 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":121,"threads_started":1}
I20260811 18:59:07.862671 29978 raft_consensus.cc:348] T 00000000000000000000000000000000 P 585cdb722eb84450938396f4093b53bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "585cdb722eb84450938396f4093b53bb" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 42205 } }
I20260811 18:59:07.862876 29978 raft_consensus.cc:374] T 00000000000000000000000000000000 P 585cdb722eb84450938396f4093b53bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:59:07.862934 29978 raft_consensus.cc:729] T 00000000000000000000000000000000 P 585cdb722eb84450938396f4093b53bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 585cdb722eb84450938396f4093b53bb, State: Initialized, Role: FOLLOWER
I20260811 18:59:07.863335 29978 consensus_queue.cc:260] T 00000000000000000000000000000000 P 585cdb722eb84450938396f4093b53bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "585cdb722eb84450938396f4093b53bb" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 42205 } }
I20260811 18:59:07.863474 29978 raft_consensus.cc:388] T 00000000000000000000000000000000 P 585cdb722eb84450938396f4093b53bb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:59:07.863534 29978 raft_consensus.cc:482] T 00000000000000000000000000000000 P 585cdb722eb84450938396f4093b53bb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:59:07.863621 29978 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 585cdb722eb84450938396f4093b53bb [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:59:07.864511 29978 raft_consensus.cc:504] T 00000000000000000000000000000000 P 585cdb722eb84450938396f4093b53bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "585cdb722eb84450938396f4093b53bb" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 42205 } }
I20260811 18:59:07.864818 29978 leader_election.cc:302] T 00000000000000000000000000000000 P 585cdb722eb84450938396f4093b53bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 585cdb722eb84450938396f4093b53bb; no voters:
I20260811 18:59:07.865131 29978 leader_election.cc:288] T 00000000000000000000000000000000 P 585cdb722eb84450938396f4093b53bb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:59:07.865248 29991 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 585cdb722eb84450938396f4093b53bb [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:59:07.865446 29991 raft_consensus.cc:686] T 00000000000000000000000000000000 P 585cdb722eb84450938396f4093b53bb [term 1 LEADER]: Becoming Leader. State: Replica: 585cdb722eb84450938396f4093b53bb, State: Running, Role: LEADER
I20260811 18:59:07.865792 29991 consensus_queue.cc:237] T 00000000000000000000000000000000 P 585cdb722eb84450938396f4093b53bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "585cdb722eb84450938396f4093b53bb" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 42205 } }
I20260811 18:59:07.866412 29978 sys_catalog.cc:564] T 00000000000000000000000000000000 P 585cdb722eb84450938396f4093b53bb [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:59:07.867612 29996 sys_catalog.cc:455] T 00000000000000000000000000000000 P 585cdb722eb84450938396f4093b53bb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 585cdb722eb84450938396f4093b53bb. Latest consensus state: current_term: 1 leader_uuid: "585cdb722eb84450938396f4093b53bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "585cdb722eb84450938396f4093b53bb" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 42205 } } }
I20260811 18:59:07.867623 29995 sys_catalog.cc:455] T 00000000000000000000000000000000 P 585cdb722eb84450938396f4093b53bb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "585cdb722eb84450938396f4093b53bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "585cdb722eb84450938396f4093b53bb" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 42205 } } }
I20260811 18:59:07.867797 29995 sys_catalog.cc:458] T 00000000000000000000000000000000 P 585cdb722eb84450938396f4093b53bb [sys.catalog]: This master's current role is: LEADER
I20260811 18:59:07.868567 29996 sys_catalog.cc:458] T 00000000000000000000000000000000 P 585cdb722eb84450938396f4093b53bb [sys.catalog]: This master's current role is: LEADER
W20260811 18:59:07.870967 30019 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 585cdb722eb84450938396f4093b53bb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:59:07.871031 30019 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:59:07.871114 30009 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:59:07.872025 30009 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:59:07.877002 30009 catalog_manager.cc:1347] Generated new cluster ID: 14fd56c4c039430c8936e0efcca660c8
I20260811 18:59:07.877060 30009 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:59:07.889662 30009 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:59:07.890614 30009 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:59:07.899081 30009 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 585cdb722eb84450938396f4093b53bb: Generated new TSK 0
I20260811 18:59:07.899712 30009 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.180.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.180.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.180.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.129.keytab.
Entry for principal kudu/127.27.180.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.129.keytab.
I20260811 18:59:07.928184 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474496157106-28370-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474496157106-28370-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474496157106-28370-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474496157106-28370-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.180.129:0
--local_ip_for_outbound_sockets=127.27.180.129
--webserver_interface=127.27.180.129
--webserver_port=0
--tserver_master_addrs=127.27.180.190:42205
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.129.keytab
--principal=kudu/127.27.180.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:34457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:59:08.038901 30034 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:59:08.039145 30034 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:59:08.039187 30034 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:59:08.043227 30034 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:59:08.043298 30034 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:59:08.043368 30034 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.180.129
I20260811 18:59:08.048447 30034 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:34457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474496157106-28370-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474496157106-28370-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.129.keytab
--principal=kudu/127.27.180.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.180.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474496157106-28370-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.180.129
--webserver_port=0
--tserver_master_addrs=127.27.180.190:42205
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30034
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.180.129
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474496157106-28370-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:59:08.049425 30034 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:59:08.050369 30034 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:59:08.059688 30047 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:08.059684 30042 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:08.059681 30043 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:59:08.059935 30034 server_base.cc:1034] running on GCE node
I20260811 18:59:08.060308 30034 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:59:08.060921 30034 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:59:08.062105 30034 hybrid_clock.cc:648] HybridClock initialized: now 1786474748062097 us; error 30 us; skew 500 ppm
Aug 11 18:59:08 dist-test-slave-2j7r krb5kdc[29821](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474748, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:59:08.064841 30034 init.cc:377] Logged in from keytab as kudu/127.27.180.129@KRBTEST.COM (short username kudu)
I20260811 18:59:08.065732 30034 webserver.cc:460] Webserver started at http://127.27.180.129:35251/ using document root <none> and password file <none>
I20260811 18:59:08.066278 30034 fs_manager.cc:359] Metadata directory not provided
I20260811 18:59:08.066344 30034 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:59:08.066529 30034 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:59:08.068571 30034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474496157106-28370-0/minicluster-data/ts-0/data/instance:
uuid: "22661a192b9a43f4b1a6c94cbaf0fbea"
format_stamp: "Formatted at 2026-08-11 18:59:08 on dist-test-slave-2j7r"
server_key: "e106a0295f76ac2410b17c32dda1be63"
server_key_iv: "6a67880a603475b2be8d7a0bd4a13508"
server_key_version: "encryptionkey@0"
I20260811 18:59:08.069161 30034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474496157106-28370-0/minicluster-data/ts-0/wal/instance:
uuid: "22661a192b9a43f4b1a6c94cbaf0fbea"
format_stamp: "Formatted at 2026-08-11 18:59:08 on dist-test-slave-2j7r"
server_key: "e106a0295f76ac2410b17c32dda1be63"
server_key_iv: "6a67880a603475b2be8d7a0bd4a13508"
server_key_version: "encryptionkey@0"
I20260811 18:59:08.072657 30034 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260811 18:59:08.075063 30062 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:08.076294 30034 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 18:59:08.076409 30034 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474496157106-28370-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474496157106-28370-0/minicluster-data/ts-0/wal
uuid: "22661a192b9a43f4b1a6c94cbaf0fbea"
format_stamp: "Formatted at 2026-08-11 18:59:08 on dist-test-slave-2j7r"
server_key: "e106a0295f76ac2410b17c32dda1be63"
server_key_iv: "6a67880a603475b2be8d7a0bd4a13508"
server_key_version: "encryptionkey@0"
I20260811 18:59:08.076508 30034 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474496157106-28370-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474496157106-28370-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474496157106-28370-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:59:08.092541 30034 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:59:08.095207 30034 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:59:08.095376 30034 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:59:08.095921 30034 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:59:08.096704 30034 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:59:08.096756 30034 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:08.096813 30034 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:59:08.096838 30034 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:08.105062 30034 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.129:45379
I20260811 18:59:08.105140 30276 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.129:45379 every 8 connection(s)
I20260811 18:59:08.105952 30034 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474496157106-28370-0/minicluster-data/ts-0/data/info.pb
I20260811 18:59:08.114449 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 30034
Aug 11 18:59:08 dist-test-slave-2j7r krb5kdc[29821](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474748, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.129@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM
I20260811 18:59:08.114571 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474496157106-28370-0/minicluster-data/ts-0/wal/instance
I20260811 18:59:08.114843 28370 external_mini_cluster.cc:1428] Setting key cb2c8a03755c860e3a9b5618f78b9449
I20260811 18:59:08.118309 29982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:08.107590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.129:52605 (local address 127.27.180.190:42205)
0811 18:59:08.107716 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:08.107720 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:08.108499 (+ 779us) server_negotiation.cc:407] Connection header received
0811 18:59:08.109793 (+ 1294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:08.109796 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:08.109845 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:08.109916 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:08.110944 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:08.111468 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:08.112504 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:08.112728 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:08.112800 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:08.115051 (+ 2251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:08.115054 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:08.115056 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:59:08.115080 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:08.116386 (+ 1306us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:08.116846 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:08.116849 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:08.116851 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:08.116922 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:08.117285 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:08.117288 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:08.117289 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:08.117458 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:08.117544 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:08.118014 (+ 470us) server_negotiation.cc:299] Negotiation successful
0811 18:59:08.118139 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 18:59:08.119056 30280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:08.107914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:42205 (local address 127.27.180.129:52605)
0811 18:59:08.108360 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:08.108401 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 18:59:08.109604 (+ 1203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:59:08.110056 (+ 452us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:59:08.110062 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:59:08.110389 (+ 327us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:59:08.110828 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:08.110841 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:08.111558 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:08.111560 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:08.112414 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:08.112423 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:08.112835 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:08.112837 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:08.112968 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:08.113467 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:59:08.113483 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:59:08.114897 (+ 1414us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:59:08.116492 (+ 1595us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:59:08.116497 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:59:08.116510 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:59:08.116747 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:59:08.117003 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:59:08.117006 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:59:08.117008 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:59:08.117194 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:59:08.117556 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:59:08.117563 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:59:08.117791 (+ 228us) client_negotiation.cc:769] Sending connection context
0811 18:59:08.117938 (+ 147us) client_negotiation.cc:240] Negotiation successful
0811 18:59:08.118253 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":131,"threads_started":1}
I20260811 18:59:08.120159 30277 heartbeater.cc:347] Connected to a master server at 127.27.180.190:42205
I20260811 18:59:08.120397 30277 heartbeater.cc:464] Registering TS with master...
I20260811 18:59:08.120851 30277 heartbeater.cc:511] Master 127.27.180.190:42205 requested a full tablet report, sending...
I20260811 18:59:08.122464 29905 ts_manager.cc:194] Registered new tserver with Master: 22661a192b9a43f4b1a6c94cbaf0fbea (127.27.180.129:45379)
I20260811 18:59:08.124001 29905 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.180.129@KRBTEST.COM'} at 127.27.180.129:52605
WARNING: no policy specified for kudu/127.27.180.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.180.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.180.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.130.keytab.
Entry for principal kudu/127.27.180.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.130.keytab.
I20260811 18:59:08.141464 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474496157106-28370-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474496157106-28370-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474496157106-28370-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474496157106-28370-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.180.130:0
--local_ip_for_outbound_sockets=127.27.180.130
--webserver_interface=127.27.180.130
--webserver_port=0
--tserver_master_addrs=127.27.180.190:42205
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.130.keytab
--principal=kudu/127.27.180.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:34457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:59:08.250376 30285 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:59:08.250618 30285 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:59:08.250659 30285 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:59:08.254596 30285 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:59:08.254643 30285 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:59:08.254705 30285 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.180.130
I20260811 18:59:08.259955 30285 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:34457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474496157106-28370-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474496157106-28370-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.130.keytab
--principal=kudu/127.27.180.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.180.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474496157106-28370-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.180.130
--webserver_port=0
--tserver_master_addrs=127.27.180.190:42205
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30285
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.180.130
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474496157106-28370-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:59:08.260910 30285 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:59:08.261900 30285 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:59:08.271275 30300 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:08.271301 30304 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:59:08.271400 30285 server_base.cc:1034] running on GCE node
W20260811 18:59:08.271333 30296 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:59:08.271900 30285 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:59:08.272481 30285 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:59:08.273625 30285 hybrid_clock.cc:648] HybridClock initialized: now 1786474748273601 us; error 36 us; skew 500 ppm
Aug 11 18:59:08 dist-test-slave-2j7r krb5kdc[29821](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474748, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:59:08.276070 30285 init.cc:377] Logged in from keytab as kudu/127.27.180.130@KRBTEST.COM (short username kudu)
I20260811 18:59:08.276891 30285 webserver.cc:460] Webserver started at http://127.27.180.130:36593/ using document root <none> and password file <none>
I20260811 18:59:08.277444 30285 fs_manager.cc:359] Metadata directory not provided
I20260811 18:59:08.277513 30285 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:59:08.277707 30285 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:59:08.279670 30285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474496157106-28370-0/minicluster-data/ts-1/data/instance:
uuid: "f2577a7f24fe42e0aa0cd7ba93918c1c"
format_stamp: "Formatted at 2026-08-11 18:59:08 on dist-test-slave-2j7r"
server_key: "04334031766f53f99214c758a8ad006b"
server_key_iv: "a082c667ee52131516d32e9e45b83d35"
server_key_version: "encryptionkey@0"
I20260811 18:59:08.280187 30285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474496157106-28370-0/minicluster-data/ts-1/wal/instance:
uuid: "f2577a7f24fe42e0aa0cd7ba93918c1c"
format_stamp: "Formatted at 2026-08-11 18:59:08 on dist-test-slave-2j7r"
server_key: "04334031766f53f99214c758a8ad006b"
server_key_iv: "a082c667ee52131516d32e9e45b83d35"
server_key_version: "encryptionkey@0"
I20260811 18:59:08.283651 30285 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260811 18:59:08.286276 30317 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:08.287746 30285 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:59:08.287858 30285 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474496157106-28370-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474496157106-28370-0/minicluster-data/ts-1/wal
uuid: "f2577a7f24fe42e0aa0cd7ba93918c1c"
format_stamp: "Formatted at 2026-08-11 18:59:08 on dist-test-slave-2j7r"
server_key: "04334031766f53f99214c758a8ad006b"
server_key_iv: "a082c667ee52131516d32e9e45b83d35"
server_key_version: "encryptionkey@0"
I20260811 18:59:08.287961 30285 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474496157106-28370-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474496157106-28370-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474496157106-28370-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:59:08.314122 30285 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:59:08.316807 30285 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:59:08.316968 30285 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:59:08.317577 30285 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:59:08.318396 30285 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:59:08.318450 30285 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:08.318491 30285 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:59:08.318521 30285 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:08.326856 30285 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.130:46387
I20260811 18:59:08.326889 30524 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.130:46387 every 8 connection(s)
I20260811 18:59:08.327669 30285 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474496157106-28370-0/minicluster-data/ts-1/data/info.pb
Aug 11 18:59:08 dist-test-slave-2j7r krb5kdc[29821](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474748, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.130@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM
I20260811 18:59:08.337353 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 30285
I20260811 18:59:08.337477 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474496157106-28370-0/minicluster-data/ts-1/wal/instance
I20260811 18:59:08.337755 28370 external_mini_cluster.cc:1428] Setting key 2e196a1b5c4579d3b83eed7282872a41
I20260811 18:59:08.340421 29982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:08.329386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.130:52487 (local address 127.27.180.190:42205)
0811 18:59:08.329504 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:08.329508 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:08.330287 (+ 779us) server_negotiation.cc:407] Connection header received
0811 18:59:08.331619 (+ 1332us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:08.331622 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:08.331680 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:08.331758 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:08.332874 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:08.333445 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:08.334539 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:08.334764 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:08.334852 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:08.337049 (+ 2197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:08.337052 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:08.337054 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:59:08.337078 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:08.338455 (+ 1377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:08.338940 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:08.338944 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:08.338946 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:08.339046 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:08.339448 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:08.339451 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:08.339452 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:08.339618 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:08.339691 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:08.340200 (+ 509us) server_negotiation.cc:299] Negotiation successful
0811 18:59:08.340300 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 18:59:08.341471 30531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:08.329695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:42205 (local address 127.27.180.130:52487)
0811 18:59:08.330139 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:08.330180 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 18:59:08.331405 (+ 1225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:59:08.331937 (+ 532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:59:08.331944 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:59:08.332277 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:59:08.332736 (+ 459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:08.332750 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:08.333567 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:08.333570 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:08.334418 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:08.334428 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:08.334901 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:08.334906 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:08.335059 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:08.335526 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:59:08.335542 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:59:08.336897 (+ 1355us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:59:08.338560 (+ 1663us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:59:08.338564 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:59:08.338574 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:59:08.338823 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:59:08.339151 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:59:08.339155 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:59:08.339157 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:59:08.339365 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:59:08.339712 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:59:08.339718 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:59:08.339987 (+ 269us) client_negotiation.cc:769] Sending connection context
0811 18:59:08.340180 (+ 193us) client_negotiation.cc:240] Negotiation successful
0811 18:59:08.340494 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":118,"threads_started":1}
I20260811 18:59:08.342501 30525 heartbeater.cc:347] Connected to a master server at 127.27.180.190:42205
I20260811 18:59:08.342713 30525 heartbeater.cc:464] Registering TS with master...
I20260811 18:59:08.343205 30525 heartbeater.cc:511] Master 127.27.180.190:42205 requested a full tablet report, sending...
I20260811 18:59:08.344270 29905 ts_manager.cc:194] Registered new tserver with Master: f2577a7f24fe42e0aa0cd7ba93918c1c (127.27.180.130:46387)
I20260811 18:59:08.345009 29905 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.180.130@KRBTEST.COM'} at 127.27.180.130:52487
WARNING: no policy specified for kudu/127.27.180.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.180.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.180.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.131.keytab.
Entry for principal kudu/127.27.180.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.131.keytab.
I20260811 18:59:08.364988 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474496157106-28370-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474496157106-28370-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474496157106-28370-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474496157106-28370-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.180.131:0
--local_ip_for_outbound_sockets=127.27.180.131
--webserver_interface=127.27.180.131
--webserver_port=0
--tserver_master_addrs=127.27.180.190:42205
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.131.keytab
--principal=kudu/127.27.180.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:34457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:59:08.472823 30540 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:59:08.473050 30540 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:59:08.473114 30540 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:59:08.476996 30540 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:59:08.477044 30540 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:59:08.477159 30540 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.180.131
I20260811 18:59:08.482331 30540 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:34457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474496157106-28370-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474496157106-28370-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.131.keytab
--principal=kudu/127.27.180.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.180.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474496157106-28370-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.180.131
--webserver_port=0
--tserver_master_addrs=127.27.180.190:42205
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30540
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.180.131
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474496157106-28370-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:59:08.483327 30540 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:59:08.484301 30540 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:59:08.493541 30554 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:08.493564 30548 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:59:08.493656 30540 server_base.cc:1034] running on GCE node
W20260811 18:59:08.493543 30551 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:59:08.494128 30540 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:59:08.494711 30540 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:59:08.495877 30540 hybrid_clock.cc:648] HybridClock initialized: now 1786474748495863 us; error 26 us; skew 500 ppm
Aug 11 18:59:08 dist-test-slave-2j7r krb5kdc[29821](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474748, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:59:08.498531 30540 init.cc:377] Logged in from keytab as kudu/127.27.180.131@KRBTEST.COM (short username kudu)
I20260811 18:59:08.499301 30540 webserver.cc:460] Webserver started at http://127.27.180.131:35205/ using document root <none> and password file <none>
I20260811 18:59:08.499806 30540 fs_manager.cc:359] Metadata directory not provided
I20260811 18:59:08.499863 30540 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:59:08.500053 30540 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:59:08.502039 30540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474496157106-28370-0/minicluster-data/ts-2/data/instance:
uuid: "9f5b244f618b4ad1a251196f3df103a7"
format_stamp: "Formatted at 2026-08-11 18:59:08 on dist-test-slave-2j7r"
server_key: "524a8edfeae366838449b7b97d24b5a5"
server_key_iv: "dfd7539cb60a5304f07567a133748dc9"
server_key_version: "encryptionkey@0"
I20260811 18:59:08.502563 30540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474496157106-28370-0/minicluster-data/ts-2/wal/instance:
uuid: "9f5b244f618b4ad1a251196f3df103a7"
format_stamp: "Formatted at 2026-08-11 18:59:08 on dist-test-slave-2j7r"
server_key: "524a8edfeae366838449b7b97d24b5a5"
server_key_iv: "dfd7539cb60a5304f07567a133748dc9"
server_key_version: "encryptionkey@0"
I20260811 18:59:08.506482 30540 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 18:59:08.508756 30566 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:08.509951 30540 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 18:59:08.510067 30540 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474496157106-28370-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474496157106-28370-0/minicluster-data/ts-2/wal
uuid: "9f5b244f618b4ad1a251196f3df103a7"
format_stamp: "Formatted at 2026-08-11 18:59:08 on dist-test-slave-2j7r"
server_key: "524a8edfeae366838449b7b97d24b5a5"
server_key_iv: "dfd7539cb60a5304f07567a133748dc9"
server_key_version: "encryptionkey@0"
I20260811 18:59:08.510165 30540 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474496157106-28370-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474496157106-28370-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474496157106-28370-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:59:08.530956 30540 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:59:08.533702 30540 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:59:08.533891 30540 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:59:08.534487 30540 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:59:08.535394 30540 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:59:08.535449 30540 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:08.535507 30540 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:59:08.535539 30540 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:08.544636 30540 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.131:40181
I20260811 18:59:08.544657 30768 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.131:40181 every 8 connection(s)
I20260811 18:59:08.545594 30540 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474496157106-28370-0/minicluster-data/ts-2/data/info.pb
I20260811 18:59:08.550498 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 30540
I20260811 18:59:08.550634 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474496157106-28370-0/minicluster-data/ts-2/wal/instance
I20260811 18:59:08.550959 28370 external_mini_cluster.cc:1428] Setting key 7860a4f5c0c94ca9ae639d93570e9f8f
Aug 11 18:59:08 dist-test-slave-2j7r krb5kdc[29821](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474748, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.131@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM
I20260811 18:59:08.558763 29982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:08.547454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.131:59723 (local address 127.27.180.190:42205)
0811 18:59:08.547586 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:08.547590 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:08.548393 (+ 803us) server_negotiation.cc:407] Connection header received
0811 18:59:08.549741 (+ 1348us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:08.549744 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:08.549803 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:08.549885 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:08.551324 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:08.551952 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:08.553336 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:08.553554 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:08.553634 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:08.555847 (+ 2213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:08.555850 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:08.555852 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:59:08.555879 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:08.557201 (+ 1322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:08.557626 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:08.557628 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:08.557629 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:08.557694 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:08.557933 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:08.557935 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:08.557936 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:08.558087 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:08.558154 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:08.558551 (+ 397us) server_negotiation.cc:299] Negotiation successful
0811 18:59:08.558646 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 18:59:08.559505 30773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:08.547811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:42205 (local address 127.27.180.131:59723)
0811 18:59:08.548248 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:08.548290 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 18:59:08.549544 (+ 1254us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:59:08.550051 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:59:08.550058 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:59:08.550491 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:59:08.551138 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:08.551158 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:08.552076 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:08.552079 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:08.553235 (+ 1156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:08.553246 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:08.553667 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:08.553670 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:08.553802 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:08.554275 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:59:08.554290 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:59:08.555701 (+ 1411us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:59:08.557296 (+ 1595us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:59:08.557300 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:59:08.557310 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:59:08.557554 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:59:08.557753 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:59:08.557755 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:59:08.557756 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:59:08.557872 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:59:08.558152 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:59:08.558157 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:59:08.558375 (+ 218us) client_negotiation.cc:769] Sending connection context
0811 18:59:08.558517 (+ 142us) client_negotiation.cc:240] Negotiation successful
0811 18:59:08.558746 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":108,"threads_started":1}
I20260811 18:59:08.560498 30769 heartbeater.cc:347] Connected to a master server at 127.27.180.190:42205
I20260811 18:59:08.560693 30769 heartbeater.cc:464] Registering TS with master...
I20260811 18:59:08.561151 30769 heartbeater.cc:511] Master 127.27.180.190:42205 requested a full tablet report, sending...
I20260811 18:59:08.562101 29905 ts_manager.cc:194] Registered new tserver with Master: 9f5b244f618b4ad1a251196f3df103a7 (127.27.180.131:40181)
I20260811 18:59:08.562611 29905 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.180.131@KRBTEST.COM'} at 127.27.180.131:59723
I20260811 18:59:08.564757 28370 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 18:59:08.573323 29982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:08.566700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42460 (local address 127.27.180.190:42205)
0811 18:59:08.566854 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:08.566858 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:08.567011 (+ 153us) server_negotiation.cc:407] Connection header received
0811 18:59:08.567097 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:08.567099 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:08.567136 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:08.567193 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:08.567871 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:08.568400 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:08.569222 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:08.569436 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:08.569513 (+ 77us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:08.570771 (+ 1258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:08.570774 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:08.570776 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:59:08.570798 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:08.571878 (+ 1080us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:08.572316 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:08.572319 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:08.572320 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:08.572388 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:08.572606 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:08.572608 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:08.572609 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:08.572789 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:08.572894 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:08.573079 (+ 185us) server_negotiation.cc:299] Negotiation successful
0811 18:59:08.573176 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 18:59:08.575778 29905 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42460:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:59:08.578752 29905 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:59:08.594391 30794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:08.588924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.130:46387 (local address 127.0.0.1:35244)
0811 18:59:08.589259 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:08.589271 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 18:59:08.589352 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:59:08.590477 (+ 1125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:59:08.590480 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:59:08.590493 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:59:08.590599 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:08.590605 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:08.591788 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:08.591792 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:08.592937 (+ 1145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:08.592947 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:08.593949 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:08.593952 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:08.594036 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:08.594066 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 18:59:08.594145 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 18:59:08.594245 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"mutex_wait_us":65,"thread_start_us":103,"threads_started":1}
I20260811 18:59:08.594415 30795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:08.589118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35244 (local address 127.27.180.130:46387)
0811 18:59:08.590022 (+ 904us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:08.590034 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:08.590057 (+ 23us) server_negotiation.cc:407] Connection header received
0811 18:59:08.590131 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:08.590137 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:08.590325 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:08.590731 (+ 406us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:59:08.590854 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:08.591640 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:08.593062 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:08.593838 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:08.593956 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:08.594081 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:08.594176 (+ 95us) server_negotiation.cc:299] Negotiation successful
0811 18:59:08.594277 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":801,"thread_start_us":121,"threads_started":1}
I20260811 18:59:08.594604 30788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:08.588434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.129:45379 (local address 127.0.0.1:45094)
0811 18:59:08.588755 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:08.588804 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 18:59:08.588917 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:59:08.589766 (+ 849us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:59:08.589773 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:59:08.589798 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:59:08.589939 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:08.589947 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:08.591628 (+ 1681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:08.591632 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:08.593189 (+ 1557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:08.593201 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:08.594242 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:08.594245 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:08.594324 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:08.594345 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 18:59:08.594424 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 18:59:08.594485 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"mutex_wait_us":100,"thread_start_us":83,"threads_started":1}
I20260811 18:59:08.595088 30793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:08.588607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45094 (local address 127.27.180.129:45379)
0811 18:59:08.589231 (+ 624us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:08.589246 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:08.589281 (+ 35us) server_negotiation.cc:407] Connection header received
0811 18:59:08.589361 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:08.589366 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:08.589578 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:08.590104 (+ 526us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:59:08.590267 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:08.591455 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:08.593317 (+ 1862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:08.594119 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:08.594238 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:08.594356 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:08.594860 (+ 504us) server_negotiation.cc:299] Negotiation successful
0811 18:59:08.594957 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":515,"thread_start_us":114,"threads_started":1}
I20260811 18:59:08.595734 30796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:08.589157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.131:40181 (local address 127.0.0.1:56554)
0811 18:59:08.589667 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:08.589681 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 18:59:08.589774 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:59:08.590420 (+ 646us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:59:08.590423 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:59:08.590442 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:59:08.590583 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:08.590590 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:08.591891 (+ 1301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:08.591894 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:08.593189 (+ 1295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:08.593201 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:08.595343 (+ 2142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:08.595347 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:08.595448 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:08.595471 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 18:59:08.595547 (+ 76us) client_negotiation.cc:240] Negotiation successful
0811 18:59:08.595610 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":443,"mutex_wait_us":102,"thread_start_us":92,"threads_started":1}
I20260811 18:59:08.596339 30797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:08.589550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56554 (local address 127.27.180.131:40181)
0811 18:59:08.590016 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:08.590027 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:08.590050 (+ 23us) server_negotiation.cc:407] Connection header received
0811 18:59:08.590117 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:08.590122 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:08.590297 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:08.590438 (+ 141us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:59:08.590697 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:08.591718 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:08.594416 (+ 2698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:08.595192 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:08.595349 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:08.595462 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:08.596121 (+ 659us) server_negotiation.cc:299] Negotiation successful
0811 18:59:08.596212 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":159,"threads_started":1}
I20260811 18:59:08.597558 30403 tablet_service.cc:1467] Processing CreateTablet for tablet 3b425584482446c5a3dcbc9585e2eaaa (DEFAULT_TABLE table=test-table [id=c1673ff748f24462af54b49aa2dbc01b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:59:08.597663 30146 tablet_service.cc:1467] Processing CreateTablet for tablet 3b425584482446c5a3dcbc9585e2eaaa (DEFAULT_TABLE table=test-table [id=c1673ff748f24462af54b49aa2dbc01b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:59:08.598488 30403 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b425584482446c5a3dcbc9585e2eaaa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:59:08.598677 30146 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b425584482446c5a3dcbc9585e2eaaa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:59:08.598803 30652 tablet_service.cc:1467] Processing CreateTablet for tablet 3b425584482446c5a3dcbc9585e2eaaa (DEFAULT_TABLE table=test-table [id=c1673ff748f24462af54b49aa2dbc01b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:59:08.599735 30652 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b425584482446c5a3dcbc9585e2eaaa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:59:08.604300 30799 tablet_bootstrap.cc:492] T 3b425584482446c5a3dcbc9585e2eaaa P f2577a7f24fe42e0aa0cd7ba93918c1c: Bootstrap starting.
I20260811 18:59:08.606297 30802 tablet_bootstrap.cc:492] T 3b425584482446c5a3dcbc9585e2eaaa P 9f5b244f618b4ad1a251196f3df103a7: Bootstrap starting.
I20260811 18:59:08.606568 30799 tablet_bootstrap.cc:654] T 3b425584482446c5a3dcbc9585e2eaaa P f2577a7f24fe42e0aa0cd7ba93918c1c: Neither blocks nor log segments found. Creating new log.
I20260811 18:59:08.606992 30800 tablet_bootstrap.cc:492] T 3b425584482446c5a3dcbc9585e2eaaa P 22661a192b9a43f4b1a6c94cbaf0fbea: Bootstrap starting.
I20260811 18:59:08.607903 30799 log.cc:824] T 3b425584482446c5a3dcbc9585e2eaaa P f2577a7f24fe42e0aa0cd7ba93918c1c: Log is configured to *not* fsync() on all Append() calls
I20260811 18:59:08.608605 30802 tablet_bootstrap.cc:654] T 3b425584482446c5a3dcbc9585e2eaaa P 9f5b244f618b4ad1a251196f3df103a7: Neither blocks nor log segments found. Creating new log.
I20260811 18:59:08.608776 30800 tablet_bootstrap.cc:654] T 3b425584482446c5a3dcbc9585e2eaaa P 22661a192b9a43f4b1a6c94cbaf0fbea: Neither blocks nor log segments found. Creating new log.
I20260811 18:59:08.609824 30800 log.cc:824] T 3b425584482446c5a3dcbc9585e2eaaa P 22661a192b9a43f4b1a6c94cbaf0fbea: Log is configured to *not* fsync() on all Append() calls
I20260811 18:59:08.609862 30802 log.cc:824] T 3b425584482446c5a3dcbc9585e2eaaa P 9f5b244f618b4ad1a251196f3df103a7: Log is configured to *not* fsync() on all Append() calls
I20260811 18:59:08.610059 30799 tablet_bootstrap.cc:492] T 3b425584482446c5a3dcbc9585e2eaaa P f2577a7f24fe42e0aa0cd7ba93918c1c: No bootstrap required, opened a new log
I20260811 18:59:08.610253 30799 ts_tablet_manager.cc:1397] T 3b425584482446c5a3dcbc9585e2eaaa P f2577a7f24fe42e0aa0cd7ba93918c1c: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.004s
I20260811 18:59:08.611714 30802 tablet_bootstrap.cc:492] T 3b425584482446c5a3dcbc9585e2eaaa P 9f5b244f618b4ad1a251196f3df103a7: No bootstrap required, opened a new log
I20260811 18:59:08.611896 30802 ts_tablet_manager.cc:1397] T 3b425584482446c5a3dcbc9585e2eaaa P 9f5b244f618b4ad1a251196f3df103a7: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 18:59:08.614044 30799 raft_consensus.cc:348] T 3b425584482446c5a3dcbc9585e2eaaa P f2577a7f24fe42e0aa0cd7ba93918c1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22661a192b9a43f4b1a6c94cbaf0fbea" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 45379 } } peers { permanent_uuid: "9f5b244f618b4ad1a251196f3df103a7" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 40181 } } peers { permanent_uuid: "f2577a7f24fe42e0aa0cd7ba93918c1c" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 46387 } }
I20260811 18:59:08.614342 30799 raft_consensus.cc:374] T 3b425584482446c5a3dcbc9585e2eaaa P f2577a7f24fe42e0aa0cd7ba93918c1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:59:08.614403 30799 raft_consensus.cc:729] T 3b425584482446c5a3dcbc9585e2eaaa P f2577a7f24fe42e0aa0cd7ba93918c1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2577a7f24fe42e0aa0cd7ba93918c1c, State: Initialized, Role: FOLLOWER
I20260811 18:59:08.614547 30800 tablet_bootstrap.cc:492] T 3b425584482446c5a3dcbc9585e2eaaa P 22661a192b9a43f4b1a6c94cbaf0fbea: No bootstrap required, opened a new log
I20260811 18:59:08.614717 30800 ts_tablet_manager.cc:1397] T 3b425584482446c5a3dcbc9585e2eaaa P 22661a192b9a43f4b1a6c94cbaf0fbea: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.003s
I20260811 18:59:08.614890 30799 consensus_queue.cc:260] T 3b425584482446c5a3dcbc9585e2eaaa P f2577a7f24fe42e0aa0cd7ba93918c1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22661a192b9a43f4b1a6c94cbaf0fbea" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 45379 } } peers { permanent_uuid: "9f5b244f618b4ad1a251196f3df103a7" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 40181 } } peers { permanent_uuid: "f2577a7f24fe42e0aa0cd7ba93918c1c" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 46387 } }
I20260811 18:59:08.615567 30525 heartbeater.cc:503] Master 127.27.180.190:42205 was elected leader, sending a full tablet report...
I20260811 18:59:08.615862 30799 ts_tablet_manager.cc:1428] T 3b425584482446c5a3dcbc9585e2eaaa P f2577a7f24fe42e0aa0cd7ba93918c1c: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260811 18:59:08.616140 30802 raft_consensus.cc:348] T 3b425584482446c5a3dcbc9585e2eaaa P 9f5b244f618b4ad1a251196f3df103a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22661a192b9a43f4b1a6c94cbaf0fbea" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 45379 } } peers { permanent_uuid: "9f5b244f618b4ad1a251196f3df103a7" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 40181 } } peers { permanent_uuid: "f2577a7f24fe42e0aa0cd7ba93918c1c" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 46387 } }
I20260811 18:59:08.616382 30802 raft_consensus.cc:374] T 3b425584482446c5a3dcbc9585e2eaaa P 9f5b244f618b4ad1a251196f3df103a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:59:08.616437 30802 raft_consensus.cc:729] T 3b425584482446c5a3dcbc9585e2eaaa P 9f5b244f618b4ad1a251196f3df103a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f5b244f618b4ad1a251196f3df103a7, State: Initialized, Role: FOLLOWER
I20260811 18:59:08.616837 30802 consensus_queue.cc:260] T 3b425584482446c5a3dcbc9585e2eaaa P 9f5b244f618b4ad1a251196f3df103a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22661a192b9a43f4b1a6c94cbaf0fbea" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 45379 } } peers { permanent_uuid: "9f5b244f618b4ad1a251196f3df103a7" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 40181 } } peers { permanent_uuid: "f2577a7f24fe42e0aa0cd7ba93918c1c" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 46387 } }
I20260811 18:59:08.617434 30769 heartbeater.cc:503] Master 127.27.180.190:42205 was elected leader, sending a full tablet report...
I20260811 18:59:08.617724 30802 ts_tablet_manager.cc:1428] T 3b425584482446c5a3dcbc9585e2eaaa P 9f5b244f618b4ad1a251196f3df103a7: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
I20260811 18:59:08.618206 30800 raft_consensus.cc:348] T 3b425584482446c5a3dcbc9585e2eaaa P 22661a192b9a43f4b1a6c94cbaf0fbea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22661a192b9a43f4b1a6c94cbaf0fbea" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 45379 } } peers { permanent_uuid: "9f5b244f618b4ad1a251196f3df103a7" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 40181 } } peers { permanent_uuid: "f2577a7f24fe42e0aa0cd7ba93918c1c" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 46387 } }
I20260811 18:59:08.618525 30800 raft_consensus.cc:374] T 3b425584482446c5a3dcbc9585e2eaaa P 22661a192b9a43f4b1a6c94cbaf0fbea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:59:08.618623 30800 raft_consensus.cc:729] T 3b425584482446c5a3dcbc9585e2eaaa P 22661a192b9a43f4b1a6c94cbaf0fbea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22661a192b9a43f4b1a6c94cbaf0fbea, State: Initialized, Role: FOLLOWER
I20260811 18:59:08.619115 30800 consensus_queue.cc:260] T 3b425584482446c5a3dcbc9585e2eaaa P 22661a192b9a43f4b1a6c94cbaf0fbea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22661a192b9a43f4b1a6c94cbaf0fbea" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 45379 } } peers { permanent_uuid: "9f5b244f618b4ad1a251196f3df103a7" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 40181 } } peers { permanent_uuid: "f2577a7f24fe42e0aa0cd7ba93918c1c" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 46387 } }
I20260811 18:59:08.619802 30277 heartbeater.cc:503] Master 127.27.180.190:42205 was elected leader, sending a full tablet report...
I20260811 18:59:08.620096 30800 ts_tablet_manager.cc:1428] T 3b425584482446c5a3dcbc9585e2eaaa P 22661a192b9a43f4b1a6c94cbaf0fbea: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260811 18:59:08.725338 30813 raft_consensus.cc:482] T 3b425584482446c5a3dcbc9585e2eaaa P 9f5b244f618b4ad1a251196f3df103a7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:59:08.725556 30813 raft_consensus.cc:504] T 3b425584482446c5a3dcbc9585e2eaaa P 9f5b244f618b4ad1a251196f3df103a7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22661a192b9a43f4b1a6c94cbaf0fbea" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 45379 } } peers { permanent_uuid: "9f5b244f618b4ad1a251196f3df103a7" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 40181 } } peers { permanent_uuid: "f2577a7f24fe42e0aa0cd7ba93918c1c" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 46387 } }
I20260811 18:59:08.726712 30813 leader_election.cc:288] T 3b425584482446c5a3dcbc9585e2eaaa P 9f5b244f618b4ad1a251196f3df103a7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 22661a192b9a43f4b1a6c94cbaf0fbea (127.27.180.129:45379), f2577a7f24fe42e0aa0cd7ba93918c1c (127.27.180.130:46387)
I20260811 18:59:08.730450 30793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:08.727030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.131:59507 (local address 127.27.180.129:45379)
0811 18:59:08.727153 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:08.727158 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:08.727176 (+ 18us) server_negotiation.cc:407] Connection header received
0811 18:59:08.727349 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:08.727353 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:08.727416 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:08.727486 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:59:08.727784 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:08.728343 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:08.729488 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:08.729884 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:08.729980 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:08.730036 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:08.730253 (+ 217us) server_negotiation.cc:299] Negotiation successful
0811 18:59:08.730329 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 18:59:08.730450 30795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:08.727160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.131:43651 (local address 127.27.180.130:46387)
0811 18:59:08.727282 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:08.727287 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:08.727434 (+ 147us) server_negotiation.cc:407] Connection header received
0811 18:59:08.727553 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:08.727556 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:08.727607 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:08.727683 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:59:08.727967 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:08.728485 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:08.729514 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:08.729912 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:08.730018 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:08.730069 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:08.730243 (+ 174us) server_negotiation.cc:299] Negotiation successful
0811 18:59:08.730317 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 18:59:08.730465 30815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:08.726987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.130:46387 (local address 127.27.180.131:43651)
0811 18:59:08.727352 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:08.727365 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:59:08.727461 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:59:08.727724 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:59:08.727727 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:59:08.727745 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:59:08.727863 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:08.727869 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:08.728588 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:08.728590 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:08.729391 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:08.729400 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:08.730027 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:08.730031 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:08.730127 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:08.730148 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 18:59:08.730209 (+ 61us) client_negotiation.cc:240] Negotiation successful
0811 18:59:08.730298 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"mutex_wait_us":4,"thread_start_us":130,"threads_started":1}
I20260811 18:59:08.730466 30773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:08.726942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.129:45379 (local address 127.27.180.131:59507)
0811 18:59:08.727107 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:08.727123 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 18:59:08.727215 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:59:08.727506 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:59:08.727509 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:59:08.727529 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:59:08.727650 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:08.727658 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:08.728456 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:08.728459 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:08.729359 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:08.729370 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:08.730003 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:08.730007 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:08.730107 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:08.730153 (+ 46us) client_negotiation.cc:769] Sending connection context
0811 18:59:08.730220 (+ 67us) client_negotiation.cc:240] Negotiation successful
0811 18:59:08.730339 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":82,"mutex_wait_us":60}
I20260811 18:59:08.731132 30187 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b425584482446c5a3dcbc9585e2eaaa" candidate_uuid: "9f5b244f618b4ad1a251196f3df103a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22661a192b9a43f4b1a6c94cbaf0fbea" is_pre_election: true
I20260811 18:59:08.731130 30427 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b425584482446c5a3dcbc9585e2eaaa" candidate_uuid: "9f5b244f618b4ad1a251196f3df103a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2577a7f24fe42e0aa0cd7ba93918c1c" is_pre_election: true
I20260811 18:59:08.731444 30427 raft_consensus.cc:2454] T 3b425584482446c5a3dcbc9585e2eaaa P f2577a7f24fe42e0aa0cd7ba93918c1c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f5b244f618b4ad1a251196f3df103a7 in term 0.
I20260811 18:59:08.731444 30187 raft_consensus.cc:2454] T 3b425584482446c5a3dcbc9585e2eaaa P 22661a192b9a43f4b1a6c94cbaf0fbea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f5b244f618b4ad1a251196f3df103a7 in term 0.
I20260811 18:59:08.731935 30570 leader_election.cc:302] T 3b425584482446c5a3dcbc9585e2eaaa P 9f5b244f618b4ad1a251196f3df103a7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22661a192b9a43f4b1a6c94cbaf0fbea, 9f5b244f618b4ad1a251196f3df103a7; no voters:
I20260811 18:59:08.732213 30813 raft_consensus.cc:2781] T 3b425584482446c5a3dcbc9585e2eaaa P 9f5b244f618b4ad1a251196f3df103a7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:59:08.732316 30813 raft_consensus.cc:482] T 3b425584482446c5a3dcbc9585e2eaaa P 9f5b244f618b4ad1a251196f3df103a7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:59:08.732370 30813 raft_consensus.cc:3037] T 3b425584482446c5a3dcbc9585e2eaaa P 9f5b244f618b4ad1a251196f3df103a7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:59:08.733412 30813 raft_consensus.cc:504] T 3b425584482446c5a3dcbc9585e2eaaa P 9f5b244f618b4ad1a251196f3df103a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22661a192b9a43f4b1a6c94cbaf0fbea" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 45379 } } peers { permanent_uuid: "9f5b244f618b4ad1a251196f3df103a7" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 40181 } } peers { permanent_uuid: "f2577a7f24fe42e0aa0cd7ba93918c1c" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 46387 } }
I20260811 18:59:08.733718 30813 leader_election.cc:288] T 3b425584482446c5a3dcbc9585e2eaaa P 9f5b244f618b4ad1a251196f3df103a7 [CANDIDATE]: Term 1 election: Requested vote from peers 22661a192b9a43f4b1a6c94cbaf0fbea (127.27.180.129:45379), f2577a7f24fe42e0aa0cd7ba93918c1c (127.27.180.130:46387)
I20260811 18:59:08.734030 30427 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b425584482446c5a3dcbc9585e2eaaa" candidate_uuid: "9f5b244f618b4ad1a251196f3df103a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2577a7f24fe42e0aa0cd7ba93918c1c"
I20260811 18:59:08.734059 30187 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b425584482446c5a3dcbc9585e2eaaa" candidate_uuid: "9f5b244f618b4ad1a251196f3df103a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22661a192b9a43f4b1a6c94cbaf0fbea"
I20260811 18:59:08.734162 30187 raft_consensus.cc:3037] T 3b425584482446c5a3dcbc9585e2eaaa P 22661a192b9a43f4b1a6c94cbaf0fbea [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:59:08.734172 30427 raft_consensus.cc:3037] T 3b425584482446c5a3dcbc9585e2eaaa P f2577a7f24fe42e0aa0cd7ba93918c1c [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:59:08.735110 30187 raft_consensus.cc:2454] T 3b425584482446c5a3dcbc9585e2eaaa P 22661a192b9a43f4b1a6c94cbaf0fbea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f5b244f618b4ad1a251196f3df103a7 in term 1.
I20260811 18:59:08.735220 30427 raft_consensus.cc:2454] T 3b425584482446c5a3dcbc9585e2eaaa P f2577a7f24fe42e0aa0cd7ba93918c1c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f5b244f618b4ad1a251196f3df103a7 in term 1.
I20260811 18:59:08.735394 30570 leader_election.cc:302] T 3b425584482446c5a3dcbc9585e2eaaa P 9f5b244f618b4ad1a251196f3df103a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22661a192b9a43f4b1a6c94cbaf0fbea, 9f5b244f618b4ad1a251196f3df103a7; no voters:
I20260811 18:59:08.735595 30813 raft_consensus.cc:2781] T 3b425584482446c5a3dcbc9585e2eaaa P 9f5b244f618b4ad1a251196f3df103a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:59:08.735775 30813 raft_consensus.cc:686] T 3b425584482446c5a3dcbc9585e2eaaa P 9f5b244f618b4ad1a251196f3df103a7 [term 1 LEADER]: Becoming Leader. State: Replica: 9f5b244f618b4ad1a251196f3df103a7, State: Running, Role: LEADER
I20260811 18:59:08.736078 30813 consensus_queue.cc:237] T 3b425584482446c5a3dcbc9585e2eaaa P 9f5b244f618b4ad1a251196f3df103a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22661a192b9a43f4b1a6c94cbaf0fbea" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 45379 } } peers { permanent_uuid: "9f5b244f618b4ad1a251196f3df103a7" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 40181 } } peers { permanent_uuid: "f2577a7f24fe42e0aa0cd7ba93918c1c" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 46387 } }
I20260811 18:59:08.739104 29905 catalog_manager.cc:5495] T 3b425584482446c5a3dcbc9585e2eaaa P 9f5b244f618b4ad1a251196f3df103a7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9f5b244f618b4ad1a251196f3df103a7 (127.27.180.131). New cstate: current_term: 1 leader_uuid: "9f5b244f618b4ad1a251196f3df103a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22661a192b9a43f4b1a6c94cbaf0fbea" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 45379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9f5b244f618b4ad1a251196f3df103a7" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 40181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2577a7f24fe42e0aa0cd7ba93918c1c" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 46387 } health_report { overall_health: UNKNOWN } } }
I20260811 18:59:08.785014 30797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:08.781583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56566 (local address 127.27.180.131:40181)
0811 18:59:08.781714 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:08.781719 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:08.781734 (+ 15us) server_negotiation.cc:407] Connection header received
0811 18:59:08.781850 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:08.781852 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:08.781909 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:08.781998 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:59:08.782261 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:08.782758 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:08.783586 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:08.783798 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:08.783885 (+ 87us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:08.784189 (+ 304us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:59:08.784588 (+ 399us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:59:08.784717 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:08.784826 (+ 109us) server_negotiation.cc:299] Negotiation successful
0811 18:59:08.784868 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 18:59:08.790674 30427 raft_consensus.cc:1261] T 3b425584482446c5a3dcbc9585e2eaaa P f2577a7f24fe42e0aa0cd7ba93918c1c [term 1 FOLLOWER]: Refusing update from remote peer 9f5b244f618b4ad1a251196f3df103a7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:59:08.790705 30187 raft_consensus.cc:1261] T 3b425584482446c5a3dcbc9585e2eaaa P 22661a192b9a43f4b1a6c94cbaf0fbea [term 1 FOLLOWER]: Refusing update from remote peer 9f5b244f618b4ad1a251196f3df103a7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:59:08.791370 30816 consensus_queue.cc:1035] T 3b425584482446c5a3dcbc9585e2eaaa P 9f5b244f618b4ad1a251196f3df103a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2577a7f24fe42e0aa0cd7ba93918c1c" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 46387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:59:08.791589 30813 consensus_queue.cc:1035] T 3b425584482446c5a3dcbc9585e2eaaa P 9f5b244f618b4ad1a251196f3df103a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "22661a192b9a43f4b1a6c94cbaf0fbea" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 45379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 18:59:08.796518 30770 tablet.cc:2369] T 3b425584482446c5a3dcbc9585e2eaaa P 9f5b244f618b4ad1a251196f3df103a7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:59:08.799932 30821 mvcc.cc:205] Tried to move back new op lower bound from 7317400571039481856 to 7317400570825547776. Current Snapshot: MvccSnapshot[applied={T|T < 7317400571039481856}]
I20260811 18:59:08.807248 29905 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42460:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:59:08.807390 29905 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42460:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:59:08.809549 29905 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 585cdb722eb84450938396f4093b53bb: Sending DeleteTablet for 3 replicas of tablet 3b425584482446c5a3dcbc9585e2eaaa
I20260811 18:59:08.810233 30146 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b425584482446c5a3dcbc9585e2eaaa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:59:08 UTC) from {username='kudu', principal='kudu/127.27.180.190@KRBTEST.COM'} at 127.0.0.1:45094
I20260811 18:59:08.810231 30403 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b425584482446c5a3dcbc9585e2eaaa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:59:08 UTC) from {username='kudu', principal='kudu/127.27.180.190@KRBTEST.COM'} at 127.0.0.1:35244
I20260811 18:59:08.810714 30838 tablet_replica.cc:330] T 3b425584482446c5a3dcbc9585e2eaaa P f2577a7f24fe42e0aa0cd7ba93918c1c: stopping tablet replica
I20260811 18:59:08.810711 30652 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b425584482446c5a3dcbc9585e2eaaa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:59:08 UTC) from {username='kudu', principal='kudu/127.27.180.190@KRBTEST.COM'} at 127.0.0.1:56554
I20260811 18:59:08.811023 30838 raft_consensus.cc:2229] T 3b425584482446c5a3dcbc9585e2eaaa P f2577a7f24fe42e0aa0cd7ba93918c1c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:59:08.811190 30841 tablet_replica.cc:330] T 3b425584482446c5a3dcbc9585e2eaaa P 9f5b244f618b4ad1a251196f3df103a7: stopping tablet replica
I20260811 18:59:08.811323 30838 raft_consensus.cc:2258] T 3b425584482446c5a3dcbc9585e2eaaa P f2577a7f24fe42e0aa0cd7ba93918c1c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:59:08.811570 30841 raft_consensus.cc:2229] T 3b425584482446c5a3dcbc9585e2eaaa P 9f5b244f618b4ad1a251196f3df103a7 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:59:08.811777 30840 tablet_replica.cc:330] T 3b425584482446c5a3dcbc9585e2eaaa P 22661a192b9a43f4b1a6c94cbaf0fbea: stopping tablet replica
I20260811 18:59:08.811796 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 30034
I20260811 18:59:08.812034 30840 raft_consensus.cc:2229] T 3b425584482446c5a3dcbc9585e2eaaa P 22661a192b9a43f4b1a6c94cbaf0fbea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:59:08.812922 30838 ts_tablet_manager.cc:1905] T 3b425584482446c5a3dcbc9585e2eaaa P f2577a7f24fe42e0aa0cd7ba93918c1c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:59:08.816025 30838 ts_tablet_manager.cc:1918] T 3b425584482446c5a3dcbc9585e2eaaa P f2577a7f24fe42e0aa0cd7ba93918c1c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:59:08.816118 30838 log.cc:1196] T 3b425584482446c5a3dcbc9585e2eaaa P f2577a7f24fe42e0aa0cd7ba93918c1c: Deleting WAL directory at /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474496157106-28370-0/minicluster-data/ts-1/wal/wals/3b425584482446c5a3dcbc9585e2eaaa
I20260811 18:59:08.816426 30838 ts_tablet_manager.cc:1939] T 3b425584482446c5a3dcbc9585e2eaaa P f2577a7f24fe42e0aa0cd7ba93918c1c: Deleting consensus metadata
I20260811 18:59:08.817445 30841 raft_consensus.cc:2258] T 3b425584482446c5a3dcbc9585e2eaaa P 9f5b244f618b4ad1a251196f3df103a7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:59:08.818162 29879 catalog_manager.cc:4841] TS f2577a7f24fe42e0aa0cd7ba93918c1c (127.27.180.130:46387): tablet 3b425584482446c5a3dcbc9585e2eaaa (table test-table [id=c1673ff748f24462af54b49aa2dbc01b]) successfully deleted
I20260811 18:59:08.825871 30841 ts_tablet_manager.cc:1905] T 3b425584482446c5a3dcbc9585e2eaaa P 9f5b244f618b4ad1a251196f3df103a7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:59:08.828074 30841 ts_tablet_manager.cc:1918] T 3b425584482446c5a3dcbc9585e2eaaa P 9f5b244f618b4ad1a251196f3df103a7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:59:08.828166 30841 log.cc:1196] T 3b425584482446c5a3dcbc9585e2eaaa P 9f5b244f618b4ad1a251196f3df103a7: Deleting WAL directory at /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786474496157106-28370-0/minicluster-data/ts-2/wal/wals/3b425584482446c5a3dcbc9585e2eaaa
I20260811 18:59:08.828470 30841 ts_tablet_manager.cc:1939] T 3b425584482446c5a3dcbc9585e2eaaa P 9f5b244f618b4ad1a251196f3df103a7: Deleting consensus metadata
I20260811 18:59:08.829510 29878 catalog_manager.cc:4841] TS 9f5b244f618b4ad1a251196f3df103a7 (127.27.180.131:40181): tablet 3b425584482446c5a3dcbc9585e2eaaa (table test-table [id=c1673ff748f24462af54b49aa2dbc01b]) successfully deleted
W20260811 18:59:08.830380 29878 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.27.180.129:45379 (error 108)
I20260811 18:59:08.831192 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 30285
I20260811 18:59:08.831375 30796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:08.831076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.129:45379 (local address 127.0.0.1:45104)
0811 18:59:08.831215 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:08.831282 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.129:45379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 18:59:08.837016 29878 catalog_manager.cc:4584] TS 22661a192b9a43f4b1a6c94cbaf0fbea (127.27.180.129:45379): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3b425584482446c5a3dcbc9585e2eaaa: Network error: Client connection negotiation failed: client connection to 127.27.180.129:45379: connect: Connection refused (error 111)
I20260811 18:59:08.842307 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 30540
I20260811 18:59:08.865309 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 29844
2026-08-11T18:59:08Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2539 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-11T18:59:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:59:08Z Disabled control of system clock
I20260811 18:59:08.904844 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474496157106-28370-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474496157106-28370-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474496157106-28370-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.190:42123
--webserver_interface=127.27.180.190
--webserver_port=0
--builtin_ntp_servers=127.27.180.148:43307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.180.190:42123
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 18:59:09.011332 30849 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:59:09.011570 30849 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:59:09.011608 30849 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:59:09.015525 30849 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:59:09.015575 30849 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:59:09.015590 30849 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:59:09.015606 30849 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:59:09.015628 30849 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:59:09.020223 30849 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:43307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474496157106-28370-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474496157106-28370-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.180.190:42123
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.190:42123
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.180.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30849
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474496157106-28370-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:59:09.021245 30849 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:59:09.022600 30849 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:59:09.030591 30860 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:59:09.030678 30849 server_base.cc:1034] running on GCE node
W20260811 18:59:09.030661 30864 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:09.030824 30871 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:59:09.031392 30849 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:59:09.032430 30849 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:59:09.033586 30849 hybrid_clock.cc:648] HybridClock initialized: now 1786474749033569 us; error 28 us; skew 500 ppm
I20260811 18:59:09.035225 30849 webserver.cc:460] Webserver started at http://127.27.180.190:35247/ using document root <none> and password file <none>
I20260811 18:59:09.035763 30849 fs_manager.cc:359] Metadata directory not provided
I20260811 18:59:09.035828 30849 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:59:09.036023 30849 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:59:09.037979 30849 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474496157106-28370-0/minicluster-data/master-0/data/instance:
uuid: "c61a3bc962a64764bb81fafa790d1885"
format_stamp: "Formatted at 2026-08-11 18:59:09 on dist-test-slave-2j7r"
server_key: "e68219949c34f4174affaf3864dce576"
server_key_iv: "a4768845f65cb0e0a62b2d133ce2aecb"
server_key_version: "encryptionkey@0"
I20260811 18:59:09.038503 30849 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474496157106-28370-0/minicluster-data/master-0/wal/instance:
uuid: "c61a3bc962a64764bb81fafa790d1885"
format_stamp: "Formatted at 2026-08-11 18:59:09 on dist-test-slave-2j7r"
server_key: "e68219949c34f4174affaf3864dce576"
server_key_iv: "a4768845f65cb0e0a62b2d133ce2aecb"
server_key_version: "encryptionkey@0"
I20260811 18:59:09.041924 30849 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 18:59:09.044157 30885 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:09.045291 30849 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 18:59:09.045399 30849 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474496157106-28370-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474496157106-28370-0/minicluster-data/master-0/wal
uuid: "c61a3bc962a64764bb81fafa790d1885"
format_stamp: "Formatted at 2026-08-11 18:59:09 on dist-test-slave-2j7r"
server_key: "e68219949c34f4174affaf3864dce576"
server_key_iv: "a4768845f65cb0e0a62b2d133ce2aecb"
server_key_version: "encryptionkey@0"
I20260811 18:59:09.045487 30849 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474496157106-28370-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474496157106-28370-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474496157106-28370-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:59:09.062849 30849 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:59:09.063508 30849 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:59:09.063678 30849 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:59:09.070158 30849 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.190:42123
I20260811 18:59:09.070178 30975 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.190:42123 every 8 connection(s)
I20260811 18:59:09.071017 30849 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
I20260811 18:59:09.074316 30976 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:59:09.079627 30976 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c61a3bc962a64764bb81fafa790d1885: Bootstrap starting.
I20260811 18:59:09.079939 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 30849
I20260811 18:59:09.080050 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474496157106-28370-0/minicluster-data/master-0/wal/instance
I20260811 18:59:09.080329 28370 external_mini_cluster.cc:1428] Setting key cca833beb61ede3d60d585124ef6cf5c
I20260811 18:59:09.082119 30976 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c61a3bc962a64764bb81fafa790d1885: Neither blocks nor log segments found. Creating new log.
I20260811 18:59:09.082825 30976 log.cc:824] T 00000000000000000000000000000000 P c61a3bc962a64764bb81fafa790d1885: Log is configured to *not* fsync() on all Append() calls
I20260811 18:59:09.084548 30976 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c61a3bc962a64764bb81fafa790d1885: No bootstrap required, opened a new log
I20260811 18:59:09.087935 30976 raft_consensus.cc:348] T 00000000000000000000000000000000 P c61a3bc962a64764bb81fafa790d1885 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c61a3bc962a64764bb81fafa790d1885" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 42123 } }
I20260811 18:59:09.088186 30976 raft_consensus.cc:374] T 00000000000000000000000000000000 P c61a3bc962a64764bb81fafa790d1885 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:59:09.088248 30976 raft_consensus.cc:729] T 00000000000000000000000000000000 P c61a3bc962a64764bb81fafa790d1885 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c61a3bc962a64764bb81fafa790d1885, State: Initialized, Role: FOLLOWER
I20260811 18:59:09.088663 30976 consensus_queue.cc:260] T 00000000000000000000000000000000 P c61a3bc962a64764bb81fafa790d1885 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c61a3bc962a64764bb81fafa790d1885" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 42123 } }
I20260811 18:59:09.088785 30976 raft_consensus.cc:388] T 00000000000000000000000000000000 P c61a3bc962a64764bb81fafa790d1885 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:59:09.088845 30976 raft_consensus.cc:482] T 00000000000000000000000000000000 P c61a3bc962a64764bb81fafa790d1885 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:59:09.088925 30980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:09.081607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59800 (local address 127.27.180.190:42123)
0811 18:59:09.082227 (+ 620us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:09.082244 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:09.082287 (+ 43us) server_negotiation.cc:407] Connection header received
0811 18:59:09.082871 (+ 584us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:09.082887 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:09.083276 (+ 389us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:09.083557 (+ 281us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:09.083958 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:09.084915 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:09.085815 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:09.086292 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:09.086432 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:09.087270 (+ 838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:09.087275 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:09.087288 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:59:09.087638 (+ 350us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:09.087690 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:59:09.087709 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:09.087800 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:09.087930 (+ 130us) server_negotiation.cc:299] Negotiation successful
0811 18:59:09.088159 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":130,"threads_started":1}
I20260811 18:59:09.088938 30976 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c61a3bc962a64764bb81fafa790d1885 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:59:09.089766 30976 raft_consensus.cc:504] T 00000000000000000000000000000000 P c61a3bc962a64764bb81fafa790d1885 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c61a3bc962a64764bb81fafa790d1885" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 42123 } }
I20260811 18:59:09.090103 30976 leader_election.cc:302] T 00000000000000000000000000000000 P c61a3bc962a64764bb81fafa790d1885 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c61a3bc962a64764bb81fafa790d1885; no voters:
I20260811 18:59:09.090418 30976 leader_election.cc:288] T 00000000000000000000000000000000 P c61a3bc962a64764bb81fafa790d1885 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:59:09.090492 30983 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c61a3bc962a64764bb81fafa790d1885 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:59:09.090687 30983 raft_consensus.cc:686] T 00000000000000000000000000000000 P c61a3bc962a64764bb81fafa790d1885 [term 1 LEADER]: Becoming Leader. State: Replica: c61a3bc962a64764bb81fafa790d1885, State: Running, Role: LEADER
I20260811 18:59:09.091003 30983 consensus_queue.cc:237] T 00000000000000000000000000000000 P c61a3bc962a64764bb81fafa790d1885 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c61a3bc962a64764bb81fafa790d1885" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 42123 } }
I20260811 18:59:09.091528 30976 sys_catalog.cc:564] T 00000000000000000000000000000000 P c61a3bc962a64764bb81fafa790d1885 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:59:09.092020 30983 sys_catalog.cc:455] T 00000000000000000000000000000000 P c61a3bc962a64764bb81fafa790d1885 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c61a3bc962a64764bb81fafa790d1885. Latest consensus state: current_term: 1 leader_uuid: "c61a3bc962a64764bb81fafa790d1885" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c61a3bc962a64764bb81fafa790d1885" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 42123 } } }
I20260811 18:59:09.092132 30983 sys_catalog.cc:458] T 00000000000000000000000000000000 P c61a3bc962a64764bb81fafa790d1885 [sys.catalog]: This master's current role is: LEADER
I20260811 18:59:09.092537 30993 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:59:09.092536 30986 sys_catalog.cc:455] T 00000000000000000000000000000000 P c61a3bc962a64764bb81fafa790d1885 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c61a3bc962a64764bb81fafa790d1885" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c61a3bc962a64764bb81fafa790d1885" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 42123 } } }
I20260811 18:59:09.092744 30986 sys_catalog.cc:458] T 00000000000000000000000000000000 P c61a3bc962a64764bb81fafa790d1885 [sys.catalog]: This master's current role is: LEADER
I20260811 18:59:09.095726 30993 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:59:09.100982 30993 catalog_manager.cc:1347] Generated new cluster ID: ad0245273a6a46ddae181fdbf4de40ad
I20260811 18:59:09.101058 30993 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:59:09.110446 30993 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:59:09.111333 30993 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:59:09.116916 30993 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c61a3bc962a64764bb81fafa790d1885: Generated new TSK 0
I20260811 18:59:09.117566 30993 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:59:09.120195 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474496157106-28370-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474496157106-28370-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474496157106-28370-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474496157106-28370-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.180.129:0
--local_ip_for_outbound_sockets=127.27.180.129
--webserver_interface=127.27.180.129
--webserver_port=0
--tserver_master_addrs=127.27.180.190:42123
--builtin_ntp_servers=127.27.180.148:43307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 18:59:09.232036 31030 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:59:09.232280 31030 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:59:09.232321 31030 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:59:09.236363 31030 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:59:09.236416 31030 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:59:09.236481 31030 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.180.129
I20260811 18:59:09.241195 31030 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:43307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474496157106-28370-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474496157106-28370-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.180.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474496157106-28370-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.180.129
--webserver_port=0
--tserver_master_addrs=127.27.180.190:42123
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31030
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.180.129
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474496157106-28370-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:59:09.242260 31030 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:59:09.243639 31030 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:59:09.253124 31041 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:09.253140 31040 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:09.253149 31046 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:59:09.253610 31030 server_base.cc:1034] running on GCE node
I20260811 18:59:09.253966 31030 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:59:09.254561 31030 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:59:09.255712 31030 hybrid_clock.cc:648] HybridClock initialized: now 1786474749255689 us; error 40 us; skew 500 ppm
I20260811 18:59:09.257658 31030 webserver.cc:460] Webserver started at http://127.27.180.129:35817/ using document root <none> and password file <none>
I20260811 18:59:09.258213 31030 fs_manager.cc:359] Metadata directory not provided
I20260811 18:59:09.258276 31030 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:59:09.258492 31030 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:59:09.260540 31030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474496157106-28370-0/minicluster-data/ts-0/data/instance:
uuid: "2d7330c3f61c459788041dff2bba1c01"
format_stamp: "Formatted at 2026-08-11 18:59:09 on dist-test-slave-2j7r"
server_key: "c790e6dac9ffeeaadf6fee4b544aeb23"
server_key_iv: "0a771edfbb2485f2772b0271bf158d43"
server_key_version: "encryptionkey@0"
I20260811 18:59:09.261075 31030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474496157106-28370-0/minicluster-data/ts-0/wal/instance:
uuid: "2d7330c3f61c459788041dff2bba1c01"
format_stamp: "Formatted at 2026-08-11 18:59:09 on dist-test-slave-2j7r"
server_key: "c790e6dac9ffeeaadf6fee4b544aeb23"
server_key_iv: "0a771edfbb2485f2772b0271bf158d43"
server_key_version: "encryptionkey@0"
I20260811 18:59:09.264649 31030 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260811 18:59:09.267076 31057 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:09.268224 31030 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 18:59:09.268334 31030 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474496157106-28370-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474496157106-28370-0/minicluster-data/ts-0/wal
uuid: "2d7330c3f61c459788041dff2bba1c01"
format_stamp: "Formatted at 2026-08-11 18:59:09 on dist-test-slave-2j7r"
server_key: "c790e6dac9ffeeaadf6fee4b544aeb23"
server_key_iv: "0a771edfbb2485f2772b0271bf158d43"
server_key_version: "encryptionkey@0"
I20260811 18:59:09.268432 31030 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474496157106-28370-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474496157106-28370-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474496157106-28370-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:59:09.280714 31030 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:59:09.281450 31030 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:59:09.281622 31030 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:59:09.282193 31030 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:59:09.283161 31030 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:59:09.283288 31030 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:09.283393 31030 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:59:09.283424 31030 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:09.292850 31030 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.129:46123
I20260811 18:59:09.292905 31263 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.129:46123 every 8 connection(s)
I20260811 18:59:09.293792 31030 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474496157106-28370-0/minicluster-data/ts-0/data/info.pb
I20260811 18:59:09.297380 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 31030
I20260811 18:59:09.297523 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474496157106-28370-0/minicluster-data/ts-0/wal/instance
I20260811 18:59:09.297887 28370 external_mini_cluster.cc:1428] Setting key edbaccf0e3d5c480f545c4617e60c109
I20260811 18:59:09.299817 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474496157106-28370-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474496157106-28370-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474496157106-28370-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474496157106-28370-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.180.130:0
--local_ip_for_outbound_sockets=127.27.180.130
--webserver_interface=127.27.180.130
--webserver_port=0
--tserver_master_addrs=127.27.180.190:42123
--builtin_ntp_servers=127.27.180.148:43307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 18:59:09.302977 30980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:09.295513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.129:56333 (local address 127.27.180.190:42123)
0811 18:59:09.295653 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:09.295656 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:09.296655 (+ 999us) server_negotiation.cc:407] Connection header received
0811 18:59:09.298276 (+ 1621us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:09.298280 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:09.298340 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:09.298437 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:09.299415 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:09.299982 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:09.301130 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:09.301352 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:09.301441 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:09.302487 (+ 1046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:09.302490 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:09.302493 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:59:09.302504 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:09.302524 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:59:09.302532 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:09.302589 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:09.302727 (+ 138us) server_negotiation.cc:299] Negotiation successful
0811 18:59:09.302788 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 18:59:09.303820 31269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:09.295892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:42123 (local address 127.27.180.129:56333)
0811 18:59:09.296533 (+ 641us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:09.296572 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 18:59:09.298044 (+ 1472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:59:09.298609 (+ 565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:59:09.298618 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:59:09.298711 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:59:09.299243 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:09.299295 (+ 52us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:09.300114 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:09.300117 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:09.300991 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:09.301001 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:09.301494 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:09.301499 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:09.301660 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:09.302231 (+ 571us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:59:09.302233 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:59:09.302273 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:59:09.302337 (+ 64us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:59:09.302601 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:59:09.302610 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:59:09.302616 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 18:59:09.302720 (+ 104us) client_negotiation.cc:240] Negotiation successful
0811 18:59:09.302995 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":124,"threads_started":1}
I20260811 18:59:09.305030 31265 heartbeater.cc:347] Connected to a master server at 127.27.180.190:42123
I20260811 18:59:09.305298 31265 heartbeater.cc:464] Registering TS with master...
I20260811 18:59:09.305917 31265 heartbeater.cc:511] Master 127.27.180.190:42123 requested a full tablet report, sending...
I20260811 18:59:09.307406 30915 ts_manager.cc:194] Registered new tserver with Master: 2d7330c3f61c459788041dff2bba1c01 (127.27.180.129:46123)
I20260811 18:59:09.308531 30915 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.27.180.129:56333
W20260811 18:59:09.408749 31270 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:59:09.408991 31270 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:59:09.409027 31270 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:59:09.412945 31270 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:59:09.412999 31270 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:59:09.413057 31270 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.180.130
I20260811 18:59:09.417725 31270 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:43307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474496157106-28370-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474496157106-28370-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.180.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474496157106-28370-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.180.130
--webserver_port=0
--tserver_master_addrs=127.27.180.190:42123
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31270
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.180.130
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474496157106-28370-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:59:09.418735 31270 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:59:09.420089 31270 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:59:09.429415 31279 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:59:09.429522 31270 server_base.cc:1034] running on GCE node
W20260811 18:59:09.429450 31283 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:09.429415 31285 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:59:09.430028 31270 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:59:09.430609 31270 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:59:09.431777 31270 hybrid_clock.cc:648] HybridClock initialized: now 1786474749431758 us; error 30 us; skew 500 ppm
I20260811 18:59:09.433537 31270 webserver.cc:460] Webserver started at http://127.27.180.130:40633/ using document root <none> and password file <none>
I20260811 18:59:09.434062 31270 fs_manager.cc:359] Metadata directory not provided
I20260811 18:59:09.434122 31270 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:59:09.434325 31270 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:59:09.436316 31270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474496157106-28370-0/minicluster-data/ts-1/data/instance:
uuid: "3daba7475a704c9f9789c2388c4be9f6"
format_stamp: "Formatted at 2026-08-11 18:59:09 on dist-test-slave-2j7r"
server_key: "9fca9346d103c376744bac2f03889523"
server_key_iv: "03ef060120f675eb0990de803f1c1ef0"
server_key_version: "encryptionkey@0"
I20260811 18:59:09.436844 31270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474496157106-28370-0/minicluster-data/ts-1/wal/instance:
uuid: "3daba7475a704c9f9789c2388c4be9f6"
format_stamp: "Formatted at 2026-08-11 18:59:09 on dist-test-slave-2j7r"
server_key: "9fca9346d103c376744bac2f03889523"
server_key_iv: "03ef060120f675eb0990de803f1c1ef0"
server_key_version: "encryptionkey@0"
I20260811 18:59:09.440310 31270 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260811 18:59:09.442657 31293 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:09.443796 31270 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 18:59:09.443902 31270 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474496157106-28370-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474496157106-28370-0/minicluster-data/ts-1/wal
uuid: "3daba7475a704c9f9789c2388c4be9f6"
format_stamp: "Formatted at 2026-08-11 18:59:09 on dist-test-slave-2j7r"
server_key: "9fca9346d103c376744bac2f03889523"
server_key_iv: "03ef060120f675eb0990de803f1c1ef0"
server_key_version: "encryptionkey@0"
I20260811 18:59:09.443993 31270 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474496157106-28370-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474496157106-28370-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474496157106-28370-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:59:09.456281 31270 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:59:09.456887 31270 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:59:09.457046 31270 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:59:09.457607 31270 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:59:09.458516 31270 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:59:09.458562 31270 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:09.458612 31270 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:59:09.458642 31270 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:09.467160 31270 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.130:38733
I20260811 18:59:09.467211 31484 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.130:38733 every 8 connection(s)
I20260811 18:59:09.468067 31270 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474496157106-28370-0/minicluster-data/ts-1/data/info.pb
I20260811 18:59:09.476505 30980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:09.469815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.130:49757 (local address 127.27.180.190:42123)
0811 18:59:09.469953 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:09.469957 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:09.470970 (+ 1013us) server_negotiation.cc:407] Connection header received
0811 18:59:09.472352 (+ 1382us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:09.472355 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:09.472404 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:09.472476 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:09.473372 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:09.473894 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:09.474946 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:09.475153 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:09.475220 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:09.476125 (+ 905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:09.476129 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:09.476131 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:59:09.476142 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:09.476160 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:59:09.476168 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:09.476221 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:09.476342 (+ 121us) server_negotiation.cc:299] Negotiation successful
0811 18:59:09.476391 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 18:59:09.476658 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 31270
I20260811 18:59:09.476809 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474496157106-28370-0/minicluster-data/ts-1/wal/instance
I20260811 18:59:09.477212 28370 external_mini_cluster.cc:1428] Setting key b5e0b96cfb29e95c5e61860529a2bf09
I20260811 18:59:09.477620 31490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:09.470183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:42123 (local address 127.27.180.130:49757)
0811 18:59:09.470830 (+ 647us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:09.470870 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 18:59:09.472167 (+ 1297us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:59:09.472622 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:59:09.472628 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:59:09.472719 (+ 91us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:59:09.473248 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:09.473263 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:09.473990 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:09.473993 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:09.474859 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:09.474868 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:09.475256 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:09.475258 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:09.475386 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:09.475903 (+ 517us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:59:09.475904 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:59:09.475941 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:59:09.475999 (+ 58us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:59:09.476234 (+ 235us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:59:09.476241 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:59:09.476246 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 18:59:09.476339 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 18:59:09.476615 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":116,"threads_started":1}
I20260811 18:59:09.478708 31486 heartbeater.cc:347] Connected to a master server at 127.27.180.190:42123
I20260811 18:59:09.478924 31486 heartbeater.cc:464] Registering TS with master...
I20260811 18:59:09.479188 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474496157106-28370-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474496157106-28370-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474496157106-28370-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474496157106-28370-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.180.131:0
--local_ip_for_outbound_sockets=127.27.180.131
--webserver_interface=127.27.180.131
--webserver_port=0
--tserver_master_addrs=127.27.180.190:42123
--builtin_ntp_servers=127.27.180.148:43307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 18:59:09.479444 31486 heartbeater.cc:511] Master 127.27.180.190:42123 requested a full tablet report, sending...
I20260811 18:59:09.480407 30915 ts_manager.cc:194] Registered new tserver with Master: 3daba7475a704c9f9789c2388c4be9f6 (127.27.180.130:38733)
I20260811 18:59:09.481016 30915 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.27.180.130:49757
W20260811 18:59:09.589178 31491 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:59:09.589411 31491 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:59:09.589447 31491 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:59:09.593361 31491 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:59:09.593403 31491 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:59:09.593453 31491 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.180.131
I20260811 18:59:09.598019 31491 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:43307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474496157106-28370-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474496157106-28370-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.180.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474496157106-28370-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.180.131
--webserver_port=0
--tserver_master_addrs=127.27.180.190:42123
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31491
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.180.131
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474496157106-28370-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:59:09.598913 31491 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:59:09.600191 31491 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:59:09.609529 31500 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:09.609545 31502 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:59:09.609699 31491 server_base.cc:1034] running on GCE node
W20260811 18:59:09.609534 31499 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:59:09.610183 31491 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:59:09.610778 31491 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:59:09.611933 31491 hybrid_clock.cc:648] HybridClock initialized: now 1786474749611903 us; error 37 us; skew 500 ppm
I20260811 18:59:09.613628 31491 webserver.cc:460] Webserver started at http://127.27.180.131:36377/ using document root <none> and password file <none>
I20260811 18:59:09.614154 31491 fs_manager.cc:359] Metadata directory not provided
I20260811 18:59:09.614218 31491 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:59:09.614414 31491 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:59:09.616421 31491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474496157106-28370-0/minicluster-data/ts-2/data/instance:
uuid: "01e73e6c56154cf6b30fd9fe710ab6f5"
format_stamp: "Formatted at 2026-08-11 18:59:09 on dist-test-slave-2j7r"
server_key: "ffb1f023eaf589b604879d1a8010abfd"
server_key_iv: "9fefdeb3c4db497d8d73f2db43b4d369"
server_key_version: "encryptionkey@0"
I20260811 18:59:09.616995 31491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474496157106-28370-0/minicluster-data/ts-2/wal/instance:
uuid: "01e73e6c56154cf6b30fd9fe710ab6f5"
format_stamp: "Formatted at 2026-08-11 18:59:09 on dist-test-slave-2j7r"
server_key: "ffb1f023eaf589b604879d1a8010abfd"
server_key_iv: "9fefdeb3c4db497d8d73f2db43b4d369"
server_key_version: "encryptionkey@0"
I20260811 18:59:09.620414 31491 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260811 18:59:09.622641 31514 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:09.623831 31491 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 18:59:09.623922 31491 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474496157106-28370-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474496157106-28370-0/minicluster-data/ts-2/wal
uuid: "01e73e6c56154cf6b30fd9fe710ab6f5"
format_stamp: "Formatted at 2026-08-11 18:59:09 on dist-test-slave-2j7r"
server_key: "ffb1f023eaf589b604879d1a8010abfd"
server_key_iv: "9fefdeb3c4db497d8d73f2db43b4d369"
server_key_version: "encryptionkey@0"
I20260811 18:59:09.624001 31491 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474496157106-28370-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474496157106-28370-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474496157106-28370-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:59:09.634443 31491 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:59:09.634989 31491 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:59:09.635118 31491 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:59:09.635594 31491 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:59:09.636427 31491 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:59:09.636472 31491 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:09.636508 31491 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:59:09.636528 31491 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:09.644392 31491 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.131:32961
I20260811 18:59:09.644428 31717 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.131:32961 every 8 connection(s)
I20260811 18:59:09.645234 31491 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474496157106-28370-0/minicluster-data/ts-2/data/info.pb
I20260811 18:59:09.646303 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 31491
I20260811 18:59:09.646437 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474496157106-28370-0/minicluster-data/ts-2/wal/instance
I20260811 18:59:09.646716 28370 external_mini_cluster.cc:1428] Setting key d59bda09c0dfa39c2eadb730aa3a81d7
I20260811 18:59:09.654357 30980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:09.647017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.131:46631 (local address 127.27.180.190:42123)
0811 18:59:09.647150 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:09.647155 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:09.648329 (+ 1174us) server_negotiation.cc:407] Connection header received
0811 18:59:09.649960 (+ 1631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:09.649963 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:09.650020 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:09.650091 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:09.650918 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:09.651438 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:09.652544 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:09.652764 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:09.652859 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:09.653898 (+ 1039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:09.653902 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:09.653904 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:59:09.653917 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:09.653935 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:59:09.653944 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:09.654002 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:09.654149 (+ 147us) server_negotiation.cc:299] Negotiation successful
0811 18:59:09.654213 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 18:59:09.655148 31721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:09.647428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:42123 (local address 127.27.180.131:46631)
0811 18:59:09.648198 (+ 770us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:09.648249 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 18:59:09.649727 (+ 1478us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:59:09.650251 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:59:09.650257 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:59:09.650338 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:59:09.650790 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:09.650802 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:09.651552 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:09.651557 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:09.652430 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:09.652440 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:09.652899 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:09.652904 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:09.653062 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:09.653648 (+ 586us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:59:09.653649 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:59:09.653687 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:59:09.653748 (+ 61us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:59:09.654013 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:59:09.654022 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:59:09.654028 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 18:59:09.654139 (+ 111us) client_negotiation.cc:240] Negotiation successful
0811 18:59:09.654423 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":136,"threads_started":1}
I20260811 18:59:09.656109 31718 heartbeater.cc:347] Connected to a master server at 127.27.180.190:42123
I20260811 18:59:09.656323 31718 heartbeater.cc:464] Registering TS with master...
I20260811 18:59:09.656795 31718 heartbeater.cc:511] Master 127.27.180.190:42123 requested a full tablet report, sending...
I20260811 18:59:09.657768 30915 ts_manager.cc:194] Registered new tserver with Master: 01e73e6c56154cf6b30fd9fe710ab6f5 (127.27.180.131:32961)
I20260811 18:59:09.658341 30915 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.27.180.131:46631
I20260811 18:59:09.660737 28370 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 18:59:09.665820 30980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:09.662548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59806 (local address 127.27.180.190:42123)
0811 18:59:09.662676 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:09.662680 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:09.662801 (+ 121us) server_negotiation.cc:407] Connection header received
0811 18:59:09.662902 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:09.662904 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:09.662941 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:09.662994 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:09.663198 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:09.663662 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:09.664447 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:09.664641 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:09.664703 (+ 62us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:09.665440 (+ 737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:09.665443 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:09.665445 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:59:09.665456 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:09.665472 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:59:09.665480 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:09.665569 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:09.665662 (+ 93us) server_negotiation.cc:299] Negotiation successful
0811 18:59:09.665695 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 18:59:09.668179 30915 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59806:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:59:09.671023 30915 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:59:09.684453 31739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:09.678676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.131:32961 (local address 127.0.0.1:46296)
0811 18:59:09.679403 (+ 727us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:09.679447 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 18:59:09.679565 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:59:09.680556 (+ 991us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:59:09.680562 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:59:09.680583 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:59:09.680700 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:09.680707 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:09.681758 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:09.681762 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:09.683025 (+ 1263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:09.683036 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:09.683937 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:09.683941 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:09.684048 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:09.684084 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 18:59:09.684183 (+ 99us) client_negotiation.cc:240] Negotiation successful
0811 18:59:09.684314 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"mutex_wait_us":59,"spinlock_wait_cycles":768,"thread_start_us":54,"threads_started":1}
I20260811 18:59:09.684464 31738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:09.678808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46296 (local address 127.27.180.131:32961)
0811 18:59:09.680203 (+ 1395us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:09.680212 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:09.680231 (+ 19us) server_negotiation.cc:407] Connection header received
0811 18:59:09.680285 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:09.680288 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:09.680417 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:09.680545 (+ 128us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:59:09.680831 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:09.681647 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:09.683161 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:09.683826 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:09.683938 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:09.684109 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:09.684230 (+ 121us) server_negotiation.cc:299] Negotiation successful
0811 18:59:09.684321 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1300,"thread_start_us":139,"threads_started":1}
I20260811 18:59:09.685081 31736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:09.679054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.129:46123 (local address 127.0.0.1:54478)
0811 18:59:09.679961 (+ 907us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:09.679973 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 18:59:09.680040 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:59:09.680687 (+ 647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:59:09.680689 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:59:09.680709 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:59:09.680806 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:09.680812 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:09.682264 (+ 1452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:09.682267 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:09.683447 (+ 1180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:09.683458 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:09.684704 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:09.684707 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:09.684787 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:09.684808 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 18:59:09.684882 (+ 74us) client_negotiation.cc:240] Negotiation successful
0811 18:59:09.684948 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":851,"mutex_wait_us":1,"thread_start_us":88,"threads_started":1}
I20260811 18:59:09.685371 31742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:09.679451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54478 (local address 127.27.180.129:46123)
0811 18:59:09.679976 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:09.679988 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:09.680018 (+ 30us) server_negotiation.cc:407] Connection header received
0811 18:59:09.680177 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:09.680184 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:09.680400 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:09.680543 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:59:09.680939 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:09.682131 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:09.683591 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:09.684369 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:09.684458 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:09.684607 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:09.685109 (+ 502us) server_negotiation.cc:299] Negotiation successful
0811 18:59:09.685217 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":425,"thread_start_us":125,"threads_started":1}
I20260811 18:59:09.686435 31737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:09.678773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.130:38733 (local address 127.0.0.1:46026)
0811 18:59:09.679721 (+ 948us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:09.679735 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 18:59:09.679811 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:59:09.680743 (+ 932us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:59:09.680746 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:59:09.680765 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:59:09.680847 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:09.680853 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:09.683450 (+ 2597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:09.683456 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:09.684930 (+ 1474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:09.684939 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:09.686072 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:09.686076 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:09.686160 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:09.686181 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 18:59:09.686243 (+ 62us) client_negotiation.cc:240] Negotiation successful
0811 18:59:09.686311 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":486,"spinlock_wait_cycles":1280,"thread_start_us":90,"threads_started":1}
I20260811 18:59:09.686518 31740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:09.679400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46026 (local address 127.27.180.130:38733)
0811 18:59:09.680304 (+ 904us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:09.680315 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:09.680338 (+ 23us) server_negotiation.cc:407] Connection header received
0811 18:59:09.680406 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:09.680410 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:09.680570 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:09.680746 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:59:09.680955 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:09.682137 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:09.685044 (+ 2907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:09.685965 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:09.686074 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:09.686245 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:09.686291 (+ 46us) server_negotiation.cc:299] Negotiation successful
0811 18:59:09.686375 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":797,"thread_start_us":117,"threads_started":1}
I20260811 18:59:09.687474 31598 tablet_service.cc:1467] Processing CreateTablet for tablet 8890f0f1dc1b4cee9e22b06d1c745309 (DEFAULT_TABLE table=test-table [id=a92a0d714f6746f6ac4a34c9dbab45ef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:59:09.688371 31598 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8890f0f1dc1b4cee9e22b06d1c745309. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:59:09.690204 31373 tablet_service.cc:1467] Processing CreateTablet for tablet 8890f0f1dc1b4cee9e22b06d1c745309 (DEFAULT_TABLE table=test-table [id=a92a0d714f6746f6ac4a34c9dbab45ef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:59:09.691112 31373 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8890f0f1dc1b4cee9e22b06d1c745309. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:59:09.690954 31150 tablet_service.cc:1467] Processing CreateTablet for tablet 8890f0f1dc1b4cee9e22b06d1c745309 (DEFAULT_TABLE table=test-table [id=a92a0d714f6746f6ac4a34c9dbab45ef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:59:09.691893 31150 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8890f0f1dc1b4cee9e22b06d1c745309. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:59:09.696399 31745 tablet_bootstrap.cc:492] T 8890f0f1dc1b4cee9e22b06d1c745309 P 01e73e6c56154cf6b30fd9fe710ab6f5: Bootstrap starting.
I20260811 18:59:09.696933 31748 tablet_bootstrap.cc:492] T 8890f0f1dc1b4cee9e22b06d1c745309 P 3daba7475a704c9f9789c2388c4be9f6: Bootstrap starting.
I20260811 18:59:09.698522 31745 tablet_bootstrap.cc:654] T 8890f0f1dc1b4cee9e22b06d1c745309 P 01e73e6c56154cf6b30fd9fe710ab6f5: Neither blocks nor log segments found. Creating new log.
I20260811 18:59:09.698613 31749 tablet_bootstrap.cc:492] T 8890f0f1dc1b4cee9e22b06d1c745309 P 2d7330c3f61c459788041dff2bba1c01: Bootstrap starting.
I20260811 18:59:09.699186 31748 tablet_bootstrap.cc:654] T 8890f0f1dc1b4cee9e22b06d1c745309 P 3daba7475a704c9f9789c2388c4be9f6: Neither blocks nor log segments found. Creating new log.
I20260811 18:59:09.699431 31745 log.cc:824] T 8890f0f1dc1b4cee9e22b06d1c745309 P 01e73e6c56154cf6b30fd9fe710ab6f5: Log is configured to *not* fsync() on all Append() calls
I20260811 18:59:09.699873 31748 log.cc:824] T 8890f0f1dc1b4cee9e22b06d1c745309 P 3daba7475a704c9f9789c2388c4be9f6: Log is configured to *not* fsync() on all Append() calls
I20260811 18:59:09.700582 31749 tablet_bootstrap.cc:654] T 8890f0f1dc1b4cee9e22b06d1c745309 P 2d7330c3f61c459788041dff2bba1c01: Neither blocks nor log segments found. Creating new log.
I20260811 18:59:09.701409 31749 log.cc:824] T 8890f0f1dc1b4cee9e22b06d1c745309 P 2d7330c3f61c459788041dff2bba1c01: Log is configured to *not* fsync() on all Append() calls
I20260811 18:59:09.701550 31745 tablet_bootstrap.cc:492] T 8890f0f1dc1b4cee9e22b06d1c745309 P 01e73e6c56154cf6b30fd9fe710ab6f5: No bootstrap required, opened a new log
I20260811 18:59:09.701757 31748 tablet_bootstrap.cc:492] T 8890f0f1dc1b4cee9e22b06d1c745309 P 3daba7475a704c9f9789c2388c4be9f6: No bootstrap required, opened a new log
I20260811 18:59:09.701783 31745 ts_tablet_manager.cc:1397] T 8890f0f1dc1b4cee9e22b06d1c745309 P 01e73e6c56154cf6b30fd9fe710ab6f5: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 18:59:09.701965 31748 ts_tablet_manager.cc:1397] T 8890f0f1dc1b4cee9e22b06d1c745309 P 3daba7475a704c9f9789c2388c4be9f6: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.001s
I20260811 18:59:09.703879 31749 tablet_bootstrap.cc:492] T 8890f0f1dc1b4cee9e22b06d1c745309 P 2d7330c3f61c459788041dff2bba1c01: No bootstrap required, opened a new log
I20260811 18:59:09.704063 31749 ts_tablet_manager.cc:1397] T 8890f0f1dc1b4cee9e22b06d1c745309 P 2d7330c3f61c459788041dff2bba1c01: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260811 18:59:09.705065 31748 raft_consensus.cc:348] T 8890f0f1dc1b4cee9e22b06d1c745309 P 3daba7475a704c9f9789c2388c4be9f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01e73e6c56154cf6b30fd9fe710ab6f5" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 32961 } } peers { permanent_uuid: "3daba7475a704c9f9789c2388c4be9f6" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 38733 } } peers { permanent_uuid: "2d7330c3f61c459788041dff2bba1c01" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 46123 } }
I20260811 18:59:09.705313 31748 raft_consensus.cc:374] T 8890f0f1dc1b4cee9e22b06d1c745309 P 3daba7475a704c9f9789c2388c4be9f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:59:09.705394 31748 raft_consensus.cc:729] T 8890f0f1dc1b4cee9e22b06d1c745309 P 3daba7475a704c9f9789c2388c4be9f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3daba7475a704c9f9789c2388c4be9f6, State: Initialized, Role: FOLLOWER
I20260811 18:59:09.705797 31748 consensus_queue.cc:260] T 8890f0f1dc1b4cee9e22b06d1c745309 P 3daba7475a704c9f9789c2388c4be9f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01e73e6c56154cf6b30fd9fe710ab6f5" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 32961 } } peers { permanent_uuid: "3daba7475a704c9f9789c2388c4be9f6" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 38733 } } peers { permanent_uuid: "2d7330c3f61c459788041dff2bba1c01" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 46123 } }
I20260811 18:59:09.705883 31745 raft_consensus.cc:348] T 8890f0f1dc1b4cee9e22b06d1c745309 P 01e73e6c56154cf6b30fd9fe710ab6f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01e73e6c56154cf6b30fd9fe710ab6f5" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 32961 } } peers { permanent_uuid: "3daba7475a704c9f9789c2388c4be9f6" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 38733 } } peers { permanent_uuid: "2d7330c3f61c459788041dff2bba1c01" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 46123 } }
I20260811 18:59:09.706207 31745 raft_consensus.cc:374] T 8890f0f1dc1b4cee9e22b06d1c745309 P 01e73e6c56154cf6b30fd9fe710ab6f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:59:09.706297 31745 raft_consensus.cc:729] T 8890f0f1dc1b4cee9e22b06d1c745309 P 01e73e6c56154cf6b30fd9fe710ab6f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01e73e6c56154cf6b30fd9fe710ab6f5, State: Initialized, Role: FOLLOWER
I20260811 18:59:09.706398 31486 heartbeater.cc:503] Master 127.27.180.190:42123 was elected leader, sending a full tablet report...
I20260811 18:59:09.706727 31745 consensus_queue.cc:260] T 8890f0f1dc1b4cee9e22b06d1c745309 P 01e73e6c56154cf6b30fd9fe710ab6f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01e73e6c56154cf6b30fd9fe710ab6f5" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 32961 } } peers { permanent_uuid: "3daba7475a704c9f9789c2388c4be9f6" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 38733 } } peers { permanent_uuid: "2d7330c3f61c459788041dff2bba1c01" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 46123 } }
I20260811 18:59:09.707442 31718 heartbeater.cc:503] Master 127.27.180.190:42123 was elected leader, sending a full tablet report...
I20260811 18:59:09.707674 31745 ts_tablet_manager.cc:1428] T 8890f0f1dc1b4cee9e22b06d1c745309 P 01e73e6c56154cf6b30fd9fe710ab6f5: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260811 18:59:09.707731 31749 raft_consensus.cc:348] T 8890f0f1dc1b4cee9e22b06d1c745309 P 2d7330c3f61c459788041dff2bba1c01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01e73e6c56154cf6b30fd9fe710ab6f5" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 32961 } } peers { permanent_uuid: "3daba7475a704c9f9789c2388c4be9f6" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 38733 } } peers { permanent_uuid: "2d7330c3f61c459788041dff2bba1c01" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 46123 } }
I20260811 18:59:09.707973 31749 raft_consensus.cc:374] T 8890f0f1dc1b4cee9e22b06d1c745309 P 2d7330c3f61c459788041dff2bba1c01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:59:09.708056 31749 raft_consensus.cc:729] T 8890f0f1dc1b4cee9e22b06d1c745309 P 2d7330c3f61c459788041dff2bba1c01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d7330c3f61c459788041dff2bba1c01, State: Initialized, Role: FOLLOWER
I20260811 18:59:09.708456 31749 consensus_queue.cc:260] T 8890f0f1dc1b4cee9e22b06d1c745309 P 2d7330c3f61c459788041dff2bba1c01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01e73e6c56154cf6b30fd9fe710ab6f5" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 32961 } } peers { permanent_uuid: "3daba7475a704c9f9789c2388c4be9f6" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 38733 } } peers { permanent_uuid: "2d7330c3f61c459788041dff2bba1c01" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 46123 } }
I20260811 18:59:09.708746 31748 ts_tablet_manager.cc:1428] T 8890f0f1dc1b4cee9e22b06d1c745309 P 3daba7475a704c9f9789c2388c4be9f6: Time spent starting tablet: real 0.007s user 0.000s sys 0.005s
I20260811 18:59:09.709468 31749 ts_tablet_manager.cc:1428] T 8890f0f1dc1b4cee9e22b06d1c745309 P 2d7330c3f61c459788041dff2bba1c01: Time spent starting tablet: real 0.005s user 0.007s sys 0.000s
I20260811 18:59:09.709751 31265 heartbeater.cc:503] Master 127.27.180.190:42123 was elected leader, sending a full tablet report...
W20260811 18:59:09.718921 31489 tablet.cc:2369] T 8890f0f1dc1b4cee9e22b06d1c745309 P 3daba7475a704c9f9789c2388c4be9f6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 18:59:09.794931 31267 tablet.cc:2369] T 8890f0f1dc1b4cee9e22b06d1c745309 P 2d7330c3f61c459788041dff2bba1c01: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:59:09.826385 31758 raft_consensus.cc:482] T 8890f0f1dc1b4cee9e22b06d1c745309 P 3daba7475a704c9f9789c2388c4be9f6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:59:09.826610 31758 raft_consensus.cc:504] T 8890f0f1dc1b4cee9e22b06d1c745309 P 3daba7475a704c9f9789c2388c4be9f6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01e73e6c56154cf6b30fd9fe710ab6f5" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 32961 } } peers { permanent_uuid: "3daba7475a704c9f9789c2388c4be9f6" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 38733 } } peers { permanent_uuid: "2d7330c3f61c459788041dff2bba1c01" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 46123 } }
I20260811 18:59:09.827824 31758 leader_election.cc:288] T 8890f0f1dc1b4cee9e22b06d1c745309 P 3daba7475a704c9f9789c2388c4be9f6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01e73e6c56154cf6b30fd9fe710ab6f5 (127.27.180.131:32961), 2d7330c3f61c459788041dff2bba1c01 (127.27.180.129:46123)
I20260811 18:59:09.831971 31490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:09.828071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.129:46123 (local address 127.27.180.130:54877)
0811 18:59:09.828177 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:09.828191 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 18:59:09.828283 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:59:09.828577 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:59:09.828580 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:59:09.828601 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:59:09.828720 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:09.828727 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:09.829719 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:09.829722 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:09.830886 (+ 1164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:09.830898 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:09.831616 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:09.831618 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:09.831693 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:09.831714 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 18:59:09.831773 (+ 59us) client_negotiation.cc:240] Negotiation successful
0811 18:59:09.831829 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25,"mutex_wait_us":1}
I20260811 18:59:09.832017 31742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:09.828178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.130:54877 (local address 127.27.180.129:46123)
0811 18:59:09.828297 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:09.828301 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:09.828345 (+ 44us) server_negotiation.cc:407] Connection header received
0811 18:59:09.828422 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:09.828425 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:09.828479 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:09.828558 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:59:09.828845 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:09.829598 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:09.831012 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:09.831507 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:09.831621 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:09.831675 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:09.831803 (+ 128us) server_negotiation.cc:299] Negotiation successful
0811 18:59:09.831888 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260811 18:59:09.832363 31738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:09.828140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.130:59267 (local address 127.27.180.131:32961)
0811 18:59:09.828291 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:09.828296 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:09.828676 (+ 380us) server_negotiation.cc:407] Connection header received
0811 18:59:09.828879 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:09.828883 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:09.828929 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:09.828990 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:59:09.829311 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:09.829919 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:09.831238 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:09.831803 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:09.831914 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:09.831966 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:09.832186 (+ 220us) server_negotiation.cc:299] Negotiation successful
0811 18:59:09.832257 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 18:59:09.832610 31187 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8890f0f1dc1b4cee9e22b06d1c745309" candidate_uuid: "3daba7475a704c9f9789c2388c4be9f6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d7330c3f61c459788041dff2bba1c01" is_pre_election: true
I20260811 18:59:09.832710 31761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:09.828075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.131:32961 (local address 127.27.180.130:59267)
0811 18:59:09.828593 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:09.828605 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 18:59:09.828736 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:59:09.829018 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:59:09.829021 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:59:09.829039 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:59:09.829185 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:09.829192 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:09.830029 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:09.830032 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:09.831131 (+ 1099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:09.831139 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:09.831930 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:09.831934 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:09.832039 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:09.832063 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 18:59:09.832498 (+ 435us) client_negotiation.cc:240] Negotiation successful
0811 18:59:09.832575 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"mutex_wait_us":178,"thread_start_us":136,"threads_started":1}
I20260811 18:59:09.832916 31187 raft_consensus.cc:2454] T 8890f0f1dc1b4cee9e22b06d1c745309 P 2d7330c3f61c459788041dff2bba1c01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3daba7475a704c9f9789c2388c4be9f6 in term 0.
I20260811 18:59:09.833308 31632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8890f0f1dc1b4cee9e22b06d1c745309" candidate_uuid: "3daba7475a704c9f9789c2388c4be9f6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01e73e6c56154cf6b30fd9fe710ab6f5" is_pre_election: true
I20260811 18:59:09.833407 31298 leader_election.cc:302] T 8890f0f1dc1b4cee9e22b06d1c745309 P 3daba7475a704c9f9789c2388c4be9f6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d7330c3f61c459788041dff2bba1c01, 3daba7475a704c9f9789c2388c4be9f6; no voters:
I20260811 18:59:09.833559 31632 raft_consensus.cc:2454] T 8890f0f1dc1b4cee9e22b06d1c745309 P 01e73e6c56154cf6b30fd9fe710ab6f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3daba7475a704c9f9789c2388c4be9f6 in term 0.
I20260811 18:59:09.833673 31758 raft_consensus.cc:2781] T 8890f0f1dc1b4cee9e22b06d1c745309 P 3daba7475a704c9f9789c2388c4be9f6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:59:09.833755 31758 raft_consensus.cc:482] T 8890f0f1dc1b4cee9e22b06d1c745309 P 3daba7475a704c9f9789c2388c4be9f6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:59:09.833802 31758 raft_consensus.cc:3037] T 8890f0f1dc1b4cee9e22b06d1c745309 P 3daba7475a704c9f9789c2388c4be9f6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:59:09.834764 31758 raft_consensus.cc:504] T 8890f0f1dc1b4cee9e22b06d1c745309 P 3daba7475a704c9f9789c2388c4be9f6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01e73e6c56154cf6b30fd9fe710ab6f5" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 32961 } } peers { permanent_uuid: "3daba7475a704c9f9789c2388c4be9f6" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 38733 } } peers { permanent_uuid: "2d7330c3f61c459788041dff2bba1c01" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 46123 } }
I20260811 18:59:09.835041 31758 leader_election.cc:288] T 8890f0f1dc1b4cee9e22b06d1c745309 P 3daba7475a704c9f9789c2388c4be9f6 [CANDIDATE]: Term 1 election: Requested vote from peers 01e73e6c56154cf6b30fd9fe710ab6f5 (127.27.180.131:32961), 2d7330c3f61c459788041dff2bba1c01 (127.27.180.129:46123)
I20260811 18:59:09.835345 31632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8890f0f1dc1b4cee9e22b06d1c745309" candidate_uuid: "3daba7475a704c9f9789c2388c4be9f6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01e73e6c56154cf6b30fd9fe710ab6f5"
I20260811 18:59:09.835386 31187 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8890f0f1dc1b4cee9e22b06d1c745309" candidate_uuid: "3daba7475a704c9f9789c2388c4be9f6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d7330c3f61c459788041dff2bba1c01"
I20260811 18:59:09.835469 31632 raft_consensus.cc:3037] T 8890f0f1dc1b4cee9e22b06d1c745309 P 01e73e6c56154cf6b30fd9fe710ab6f5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:59:09.835479 31187 raft_consensus.cc:3037] T 8890f0f1dc1b4cee9e22b06d1c745309 P 2d7330c3f61c459788041dff2bba1c01 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:59:09.836433 31187 raft_consensus.cc:2454] T 8890f0f1dc1b4cee9e22b06d1c745309 P 2d7330c3f61c459788041dff2bba1c01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3daba7475a704c9f9789c2388c4be9f6 in term 1.
I20260811 18:59:09.836535 31632 raft_consensus.cc:2454] T 8890f0f1dc1b4cee9e22b06d1c745309 P 01e73e6c56154cf6b30fd9fe710ab6f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3daba7475a704c9f9789c2388c4be9f6 in term 1.
I20260811 18:59:09.836709 31298 leader_election.cc:302] T 8890f0f1dc1b4cee9e22b06d1c745309 P 3daba7475a704c9f9789c2388c4be9f6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d7330c3f61c459788041dff2bba1c01, 3daba7475a704c9f9789c2388c4be9f6; no voters:
I20260811 18:59:09.836876 31758 raft_consensus.cc:2781] T 8890f0f1dc1b4cee9e22b06d1c745309 P 3daba7475a704c9f9789c2388c4be9f6 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:59:09.837049 31758 raft_consensus.cc:686] T 8890f0f1dc1b4cee9e22b06d1c745309 P 3daba7475a704c9f9789c2388c4be9f6 [term 1 LEADER]: Becoming Leader. State: Replica: 3daba7475a704c9f9789c2388c4be9f6, State: Running, Role: LEADER
I20260811 18:59:09.837353 31758 consensus_queue.cc:237] T 8890f0f1dc1b4cee9e22b06d1c745309 P 3daba7475a704c9f9789c2388c4be9f6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01e73e6c56154cf6b30fd9fe710ab6f5" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 32961 } } peers { permanent_uuid: "3daba7475a704c9f9789c2388c4be9f6" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 38733 } } peers { permanent_uuid: "2d7330c3f61c459788041dff2bba1c01" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 46123 } }
I20260811 18:59:09.840360 30915 catalog_manager.cc:5495] T 8890f0f1dc1b4cee9e22b06d1c745309 P 3daba7475a704c9f9789c2388c4be9f6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3daba7475a704c9f9789c2388c4be9f6 (127.27.180.130). New cstate: current_term: 1 leader_uuid: "3daba7475a704c9f9789c2388c4be9f6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01e73e6c56154cf6b30fd9fe710ab6f5" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 32961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3daba7475a704c9f9789c2388c4be9f6" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 38733 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2d7330c3f61c459788041dff2bba1c01" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 46123 } health_report { overall_health: UNKNOWN } } }
I20260811 18:59:09.873215 31740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:09.869978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46036 (local address 127.27.180.130:38733)
0811 18:59:09.870095 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:09.870099 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:09.870112 (+ 13us) server_negotiation.cc:407] Connection header received
0811 18:59:09.870179 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:09.870180 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:09.870230 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:09.870316 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:59:09.870494 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:09.870969 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:09.871767 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:09.872001 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:09.872076 (+ 75us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:09.872369 (+ 293us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:59:09.872763 (+ 394us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:59:09.872873 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:09.873021 (+ 148us) server_negotiation.cc:299] Negotiation successful
0811 18:59:09.873054 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 18:59:09.878736 31187 raft_consensus.cc:1261] T 8890f0f1dc1b4cee9e22b06d1c745309 P 2d7330c3f61c459788041dff2bba1c01 [term 1 FOLLOWER]: Refusing update from remote peer 3daba7475a704c9f9789c2388c4be9f6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:59:09.878736 31632 raft_consensus.cc:1261] T 8890f0f1dc1b4cee9e22b06d1c745309 P 01e73e6c56154cf6b30fd9fe710ab6f5 [term 1 FOLLOWER]: Refusing update from remote peer 3daba7475a704c9f9789c2388c4be9f6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:59:09.879365 31762 consensus_queue.cc:1035] T 8890f0f1dc1b4cee9e22b06d1c745309 P 3daba7475a704c9f9789c2388c4be9f6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d7330c3f61c459788041dff2bba1c01" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 46123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:59:09.879536 31758 consensus_queue.cc:1035] T 8890f0f1dc1b4cee9e22b06d1c745309 P 3daba7475a704c9f9789c2388c4be9f6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01e73e6c56154cf6b30fd9fe710ab6f5" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 32961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:59:09.893709 30915 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:59806:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:59:09.893874 30915 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:59806:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:59:09.896193 30915 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c61a3bc962a64764bb81fafa790d1885: Sending DeleteTablet for 3 replicas of tablet 8890f0f1dc1b4cee9e22b06d1c745309
I20260811 18:59:09.896848 31598 tablet_service.cc:1514] Processing DeleteTablet for tablet 8890f0f1dc1b4cee9e22b06d1c745309 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:59:09 UTC) from {username='slave'} at 127.0.0.1:46296
I20260811 18:59:09.897105 31150 tablet_service.cc:1514] Processing DeleteTablet for tablet 8890f0f1dc1b4cee9e22b06d1c745309 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:59:09 UTC) from {username='slave'} at 127.0.0.1:54478
I20260811 18:59:09.897498 31792 tablet_replica.cc:330] T 8890f0f1dc1b4cee9e22b06d1c745309 P 01e73e6c56154cf6b30fd9fe710ab6f5: stopping tablet replica
I20260811 18:59:09.897483 31373 tablet_service.cc:1514] Processing DeleteTablet for tablet 8890f0f1dc1b4cee9e22b06d1c745309 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:59:09 UTC) from {username='slave'} at 127.0.0.1:46026
I20260811 18:59:09.897661 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 31030
I20260811 18:59:09.911828 31792 raft_consensus.cc:2229] T 8890f0f1dc1b4cee9e22b06d1c745309 P 01e73e6c56154cf6b30fd9fe710ab6f5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:59:09.912106 31795 tablet_replica.cc:330] T 8890f0f1dc1b4cee9e22b06d1c745309 P 3daba7475a704c9f9789c2388c4be9f6: stopping tablet replica
I20260811 18:59:09.912307 31792 raft_consensus.cc:2258] T 8890f0f1dc1b4cee9e22b06d1c745309 P 01e73e6c56154cf6b30fd9fe710ab6f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:59:09.912544 31795 raft_consensus.cc:2229] T 8890f0f1dc1b4cee9e22b06d1c745309 P 3daba7475a704c9f9789c2388c4be9f6 [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:59:09.912952 31795 raft_consensus.cc:2258] T 8890f0f1dc1b4cee9e22b06d1c745309 P 3daba7475a704c9f9789c2388c4be9f6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:59:09.915161 31792 ts_tablet_manager.cc:1905] T 8890f0f1dc1b4cee9e22b06d1c745309 P 01e73e6c56154cf6b30fd9fe710ab6f5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:59:09.917534 31792 ts_tablet_manager.cc:1918] T 8890f0f1dc1b4cee9e22b06d1c745309 P 01e73e6c56154cf6b30fd9fe710ab6f5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:59:09.917695 31792 log.cc:1196] T 8890f0f1dc1b4cee9e22b06d1c745309 P 01e73e6c56154cf6b30fd9fe710ab6f5: Deleting WAL directory at /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474496157106-28370-0/minicluster-data/ts-2/wal/wals/8890f0f1dc1b4cee9e22b06d1c745309
I20260811 18:59:09.918104 31792 ts_tablet_manager.cc:1939] T 8890f0f1dc1b4cee9e22b06d1c745309 P 01e73e6c56154cf6b30fd9fe710ab6f5: Deleting consensus metadata
I20260811 18:59:09.918514 31795 ts_tablet_manager.cc:1905] T 8890f0f1dc1b4cee9e22b06d1c745309 P 3daba7475a704c9f9789c2388c4be9f6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:59:09.919507 30886 catalog_manager.cc:4841] TS 01e73e6c56154cf6b30fd9fe710ab6f5 (127.27.180.131:32961): tablet 8890f0f1dc1b4cee9e22b06d1c745309 (table test-table [id=a92a0d714f6746f6ac4a34c9dbab45ef]) successfully deleted
I20260811 18:59:09.921176 31795 ts_tablet_manager.cc:1918] T 8890f0f1dc1b4cee9e22b06d1c745309 P 3daba7475a704c9f9789c2388c4be9f6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:59:09.921270 31795 log.cc:1196] T 8890f0f1dc1b4cee9e22b06d1c745309 P 3daba7475a704c9f9789c2388c4be9f6: Deleting WAL directory at /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786474496157106-28370-0/minicluster-data/ts-1/wal/wals/8890f0f1dc1b4cee9e22b06d1c745309
I20260811 18:59:09.921581 31795 ts_tablet_manager.cc:1939] T 8890f0f1dc1b4cee9e22b06d1c745309 P 3daba7475a704c9f9789c2388c4be9f6: Deleting consensus metadata
W20260811 18:59:09.922710 30892 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.27.180.129:46123 (error 108)
I20260811 18:59:09.922758 30887 catalog_manager.cc:4841] TS 3daba7475a704c9f9789c2388c4be9f6 (127.27.180.130:38733): tablet 8890f0f1dc1b4cee9e22b06d1c745309 (table test-table [id=a92a0d714f6746f6ac4a34c9dbab45ef]) successfully deleted
I20260811 18:59:09.923702 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 31270
I20260811 18:59:09.924009 31737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:09.923692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.129:46123 (local address 127.0.0.1:54486)
0811 18:59:09.923847 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:09.923917 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.129:46123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 18:59:09.926734 30892 catalog_manager.cc:4584] TS 2d7330c3f61c459788041dff2bba1c01 (127.27.180.129:46123): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8890f0f1dc1b4cee9e22b06d1c745309: Network error: Client connection negotiation failed: client connection to 127.27.180.129:46123: connect: Connection refused (error 111)
I20260811 18:59:09.933857 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 31491
I20260811 18:59:09.948526 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 30849
2026-08-11T18:59:09Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1083 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474496157106-28370-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:59:10 dist-test-slave-2j7r krb5kdc[31804](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:59:10 dist-test-slave-2j7r krb5kdc[31804](info): set up 2 sockets
Aug 11 18:59:10 dist-test-slave-2j7r krb5kdc[31804](info): commencing operation
krb5kdc: starting...
W20260811 18:59:11.151973 28370 mini_kdc.cc:121] Time spent starting KDC: real 1.172s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474496157106-28370-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474496157106-28370-0/krb5kdc/test-user.keytab.
Aug 11 18:59:11 dist-test-slave-2j7r krb5kdc[31804](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474751, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:59:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:59:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.180.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.180.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.180.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab.
Entry for principal kudu/127.27.180.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab.
I20260811 18:59:11.269011 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474496157106-28370-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474496157106-28370-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474496157106-28370-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.190:33077
--webserver_interface=127.27.180.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab
--principal=kudu/127.27.180.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:33269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.180.190:33077
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:59:11.377017 31832 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:59:11.377269 31832 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:59:11.377323 31832 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:59:11.381268 31832 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:59:11.381319 31832 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:59:11.381345 31832 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:59:11.381368 31832 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:59:11.381397 31832 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:59:11.386475 31832 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:33269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474496157106-28370-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474496157106-28370-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.180.190:33077
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab
--principal=kudu/127.27.180.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.190:33077
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.180.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31832
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474496157106-28370-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:59:11.387391 31832 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:59:11.388314 31832 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:59:11.396198 31855 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:11.396178 31844 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:11.396179 31848 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:59:11.396318 31832 server_base.cc:1034] running on GCE node
I20260811 18:59:11.397078 31832 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:59:11.398133 31832 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:59:11.399290 31832 hybrid_clock.cc:648] HybridClock initialized: now 1786474751399272 us; error 30 us; skew 500 ppm
Aug 11 18:59:11 dist-test-slave-2j7r krb5kdc[31804](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474751, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:59:11.402001 31832 init.cc:377] Logged in from keytab as kudu/127.27.180.190@KRBTEST.COM (short username kudu)
I20260811 18:59:11.402840 31832 webserver.cc:460] Webserver started at http://127.27.180.190:40419/ using document root <none> and password file <none>
I20260811 18:59:11.403367 31832 fs_manager.cc:359] Metadata directory not provided
I20260811 18:59:11.403434 31832 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:59:11.403653 31832 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:59:11.405692 31832 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474496157106-28370-0/minicluster-data/master-0/data/instance:
uuid: "f1d8d163358442eb9e9aa032d526a29c"
format_stamp: "Formatted at 2026-08-11 18:59:11 on dist-test-slave-2j7r"
server_key: "64e6bdc615d1251bc44f93befc6aee58"
server_key_iv: "e3cc5a533275cb4f0d754ab77851449b"
server_key_version: "encryptionkey@0"
I20260811 18:59:11.406232 31832 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474496157106-28370-0/minicluster-data/master-0/wal/instance:
uuid: "f1d8d163358442eb9e9aa032d526a29c"
format_stamp: "Formatted at 2026-08-11 18:59:11 on dist-test-slave-2j7r"
server_key: "64e6bdc615d1251bc44f93befc6aee58"
server_key_iv: "e3cc5a533275cb4f0d754ab77851449b"
server_key_version: "encryptionkey@0"
I20260811 18:59:11.409858 31832 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 18:59:11.412130 31863 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:11.413328 31832 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 18:59:11.413456 31832 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474496157106-28370-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474496157106-28370-0/minicluster-data/master-0/wal
uuid: "f1d8d163358442eb9e9aa032d526a29c"
format_stamp: "Formatted at 2026-08-11 18:59:11 on dist-test-slave-2j7r"
server_key: "64e6bdc615d1251bc44f93befc6aee58"
server_key_iv: "e3cc5a533275cb4f0d754ab77851449b"
server_key_version: "encryptionkey@0"
I20260811 18:59:11.413551 31832 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474496157106-28370-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474496157106-28370-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474496157106-28370-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:59:11.436990 31832 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:59:11.439879 31832 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:59:11.440068 31832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:59:11.447088 31967 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.190:33077 every 8 connection(s)
I20260811 18:59:11.447155 31832 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.190:33077
I20260811 18:59:11.448107 31832 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
I20260811 18:59:11.450881 31968 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:59:11.454478 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 31832
I20260811 18:59:11.454622 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474496157106-28370-0/minicluster-data/master-0/wal/instance
I20260811 18:59:11.454969 28370 external_mini_cluster.cc:1428] Setting key 4ecc97ec3ffb0f31ee65b994d640c472
I20260811 18:59:11.456683 31968 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1d8d163358442eb9e9aa032d526a29c: Bootstrap starting.
I20260811 18:59:11.459353 31968 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f1d8d163358442eb9e9aa032d526a29c: Neither blocks nor log segments found. Creating new log.
I20260811 18:59:11.460227 31968 log.cc:824] T 00000000000000000000000000000000 P f1d8d163358442eb9e9aa032d526a29c: Log is configured to *not* fsync() on all Append() calls
I20260811 18:59:11.462322 31968 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1d8d163358442eb9e9aa032d526a29c: No bootstrap required, opened a new log
Aug 11 18:59:11 dist-test-slave-2j7r krb5kdc[31804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474751, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM
I20260811 18:59:11.466164 31968 raft_consensus.cc:348] T 00000000000000000000000000000000 P f1d8d163358442eb9e9aa032d526a29c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1d8d163358442eb9e9aa032d526a29c" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 33077 } }
I20260811 18:59:11.466375 31968 raft_consensus.cc:374] T 00000000000000000000000000000000 P f1d8d163358442eb9e9aa032d526a29c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:59:11.466411 31968 raft_consensus.cc:729] T 00000000000000000000000000000000 P f1d8d163358442eb9e9aa032d526a29c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1d8d163358442eb9e9aa032d526a29c, State: Initialized, Role: FOLLOWER
I20260811 18:59:11.466811 31968 consensus_queue.cc:260] T 00000000000000000000000000000000 P f1d8d163358442eb9e9aa032d526a29c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1d8d163358442eb9e9aa032d526a29c" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 33077 } }
I20260811 18:59:11.466981 31968 raft_consensus.cc:388] T 00000000000000000000000000000000 P f1d8d163358442eb9e9aa032d526a29c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:59:11.467051 31968 raft_consensus.cc:482] T 00000000000000000000000000000000 P f1d8d163358442eb9e9aa032d526a29c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:59:11.467150 31968 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f1d8d163358442eb9e9aa032d526a29c [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:59:11.468078 31968 raft_consensus.cc:504] T 00000000000000000000000000000000 P f1d8d163358442eb9e9aa032d526a29c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1d8d163358442eb9e9aa032d526a29c" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 33077 } }
I20260811 18:59:11.468438 31968 leader_election.cc:302] T 00000000000000000000000000000000 P f1d8d163358442eb9e9aa032d526a29c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f1d8d163358442eb9e9aa032d526a29c; no voters:
I20260811 18:59:11.468802 31968 leader_election.cc:288] T 00000000000000000000000000000000 P f1d8d163358442eb9e9aa032d526a29c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:59:11.468912 31976 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f1d8d163358442eb9e9aa032d526a29c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:59:11.469151 31976 raft_consensus.cc:686] T 00000000000000000000000000000000 P f1d8d163358442eb9e9aa032d526a29c [term 1 LEADER]: Becoming Leader. State: Replica: f1d8d163358442eb9e9aa032d526a29c, State: Running, Role: LEADER
I20260811 18:59:11.469856 31972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:11.456452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55134 (local address 127.27.180.190:33077)
0811 18:59:11.456860 (+ 408us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:11.456878 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:11.456922 (+ 44us) server_negotiation.cc:407] Connection header received
0811 18:59:11.457740 (+ 818us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:11.457764 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:11.458182 (+ 418us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:11.458536 (+ 354us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:11.459190 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:11.460654 (+ 1464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:11.461633 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:11.462311 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:11.462471 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:11.465058 (+ 2587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:11.465064 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:11.465077 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:59:11.465129 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:11.466753 (+ 1624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:11.467218 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:11.467223 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:11.467230 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:11.467351 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:11.467627 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:11.467629 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:11.467630 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:11.468008 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:11.468155 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:11.468506 (+ 351us) server_negotiation.cc:299] Negotiation successful
0811 18:59:11.468791 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":258,"thread_start_us":102,"threads_started":1}
I20260811 18:59:11.469978 31968 sys_catalog.cc:564] T 00000000000000000000000000000000 P f1d8d163358442eb9e9aa032d526a29c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:59:11.470127 31976 consensus_queue.cc:237] T 00000000000000000000000000000000 P f1d8d163358442eb9e9aa032d526a29c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1d8d163358442eb9e9aa032d526a29c" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 33077 } }
I20260811 18:59:11.472419 31978 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1d8d163358442eb9e9aa032d526a29c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f1d8d163358442eb9e9aa032d526a29c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1d8d163358442eb9e9aa032d526a29c" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 33077 } } }
I20260811 18:59:11.472543 31978 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1d8d163358442eb9e9aa032d526a29c [sys.catalog]: This master's current role is: LEADER
I20260811 18:59:11.472848 31979 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1d8d163358442eb9e9aa032d526a29c [sys.catalog]: SysCatalogTable state changed. Reason: New leader f1d8d163358442eb9e9aa032d526a29c. Latest consensus state: current_term: 1 leader_uuid: "f1d8d163358442eb9e9aa032d526a29c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1d8d163358442eb9e9aa032d526a29c" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 33077 } } }
I20260811 18:59:11.472968 31979 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1d8d163358442eb9e9aa032d526a29c [sys.catalog]: This master's current role is: LEADER
W20260811 18:59:11.474537 31998 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f1d8d163358442eb9e9aa032d526a29c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:59:11.474601 31998 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:59:11.474671 32001 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:59:11.475432 32001 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:59:11.480641 32001 catalog_manager.cc:1347] Generated new cluster ID: aff46ed8e72740c38321c7cd80c8ee59
I20260811 18:59:11.480707 32001 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:59:11.507187 32001 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:59:11.508500 32001 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:59:11.523887 32001 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f1d8d163358442eb9e9aa032d526a29c: Generated new TSK 0
I20260811 18:59:11.524664 32001 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.27.180.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.180.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.180.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.129.keytab.
Entry for principal kudu/127.27.180.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.129.keytab.
I20260811 18:59:11.553952 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474496157106-28370-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474496157106-28370-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474496157106-28370-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474496157106-28370-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.180.129:0
--local_ip_for_outbound_sockets=127.27.180.129
--webserver_interface=127.27.180.129
--webserver_port=0
--tserver_master_addrs=127.27.180.190:33077
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.129.keytab
--principal=kudu/127.27.180.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:33269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:59:11.662847 32015 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:59:11.663082 32015 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:59:11.663127 32015 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:59:11.667119 32015 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:59:11.667171 32015 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:59:11.667276 32015 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.180.129
I20260811 18:59:11.672547 32015 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:33269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474496157106-28370-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474496157106-28370-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.129.keytab
--principal=kudu/127.27.180.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.180.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474496157106-28370-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.180.129
--webserver_port=0
--tserver_master_addrs=127.27.180.190:33077
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32015
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.180.129
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474496157106-28370-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:59:11.673554 32015 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:59:11.674532 32015 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 18:59:11.684253 32015 server_base.cc:1034] running on GCE node
W20260811 18:59:11.684144 32021 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:11.684139 32020 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:11.684190 32024 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:59:11.684885 32015 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:59:11.685523 32015 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:59:11.686683 32015 hybrid_clock.cc:648] HybridClock initialized: now 1786474751686656 us; error 40 us; skew 500 ppm
Aug 11 18:59:11 dist-test-slave-2j7r krb5kdc[31804](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474751, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:59:11.689314 32015 init.cc:377] Logged in from keytab as kudu/127.27.180.129@KRBTEST.COM (short username kudu)
I20260811 18:59:11.690167 32015 webserver.cc:460] Webserver started at http://127.27.180.129:41961/ using document root <none> and password file <none>
I20260811 18:59:11.690691 32015 fs_manager.cc:359] Metadata directory not provided
I20260811 18:59:11.690755 32015 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:59:11.690956 32015 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:59:11.692960 32015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474496157106-28370-0/minicluster-data/ts-0/data/instance:
uuid: "db36775e0a434ab1acd7f4fd4f9cb1a9"
format_stamp: "Formatted at 2026-08-11 18:59:11 on dist-test-slave-2j7r"
server_key: "a5f477d2cc37d7667a2946a647cb4803"
server_key_iv: "a141850e4c3c1b826ad84933e4fd9973"
server_key_version: "encryptionkey@0"
I20260811 18:59:11.693536 32015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474496157106-28370-0/minicluster-data/ts-0/wal/instance:
uuid: "db36775e0a434ab1acd7f4fd4f9cb1a9"
format_stamp: "Formatted at 2026-08-11 18:59:11 on dist-test-slave-2j7r"
server_key: "a5f477d2cc37d7667a2946a647cb4803"
server_key_iv: "a141850e4c3c1b826ad84933e4fd9973"
server_key_version: "encryptionkey@0"
I20260811 18:59:11.697137 32015 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260811 18:59:11.699442 32039 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:11.700580 32015 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260811 18:59:11.700686 32015 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474496157106-28370-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474496157106-28370-0/minicluster-data/ts-0/wal
uuid: "db36775e0a434ab1acd7f4fd4f9cb1a9"
format_stamp: "Formatted at 2026-08-11 18:59:11 on dist-test-slave-2j7r"
server_key: "a5f477d2cc37d7667a2946a647cb4803"
server_key_iv: "a141850e4c3c1b826ad84933e4fd9973"
server_key_version: "encryptionkey@0"
I20260811 18:59:11.700775 32015 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474496157106-28370-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474496157106-28370-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474496157106-28370-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:59:11.712235 32015 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:59:11.714820 32015 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:59:11.714987 32015 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:59:11.715524 32015 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:59:11.716285 32015 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:59:11.716331 32015 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:11.716403 32015 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:59:11.716461 32015 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:11.724790 32015 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.129:42995
I20260811 18:59:11.724825 32266 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.129:42995 every 8 connection(s)
I20260811 18:59:11.725611 32015 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474496157106-28370-0/minicluster-data/ts-0/data/info.pb
I20260811 18:59:11.729868 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 32015
I20260811 18:59:11.729993 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474496157106-28370-0/minicluster-data/ts-0/wal/instance
I20260811 18:59:11.730255 28370 external_mini_cluster.cc:1428] Setting key 8fde5df8e61dfd4c50036c8c6de16229
Aug 11 18:59:11 dist-test-slave-2j7r krb5kdc[31804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474751, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.129@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM
I20260811 18:59:11.739321 31972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:11.727370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.129:33337 (local address 127.27.180.190:33077)
0811 18:59:11.727502 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:11.727506 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:11.728249 (+ 743us) server_negotiation.cc:407] Connection header received
0811 18:59:11.729557 (+ 1308us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:11.729560 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:11.729617 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:11.729702 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:11.730890 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:11.731472 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:11.732605 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:11.732914 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:11.733014 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:11.735633 (+ 2619us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:11.735636 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:11.735638 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:59:11.735663 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:11.737031 (+ 1368us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:11.737686 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:11.737689 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:11.737690 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:11.737780 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:11.738173 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:11.738175 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:11.738176 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:11.738337 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:11.738408 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:11.739082 (+ 674us) server_negotiation.cc:299] Negotiation successful
0811 18:59:11.739188 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 18:59:11.740398 32272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:11.727697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33077 (local address 127.27.180.129:33337)
0811 18:59:11.728118 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:11.728157 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 18:59:11.729346 (+ 1189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:59:11.729873 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:59:11.729880 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:59:11.730256 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:59:11.730756 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:11.730770 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:11.731596 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:11.731599 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:11.732465 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:11.732475 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:11.733051 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:11.733054 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:11.733235 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:11.733785 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:59:11.733808 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:59:11.735452 (+ 1644us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:59:11.737178 (+ 1726us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:59:11.737186 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:59:11.737202 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:59:11.737576 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:59:11.737873 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:59:11.737876 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:59:11.737878 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:59:11.738072 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:59:11.738443 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:59:11.738450 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:59:11.738788 (+ 338us) client_negotiation.cc:769] Sending connection context
0811 18:59:11.739021 (+ 233us) client_negotiation.cc:240] Negotiation successful
0811 18:59:11.739369 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":103,"threads_started":1}
WARNING: no policy specified for kudu/127.27.180.130@KRBTEST.COM; defaulting to no policy
I20260811 18:59:11.741679 32270 heartbeater.cc:347] Connected to a master server at 127.27.180.190:33077
I20260811 18:59:11.741904 32270 heartbeater.cc:464] Registering TS with master...
I20260811 18:59:11.742372 32270 heartbeater.cc:511] Master 127.27.180.190:33077 requested a full tablet report, sending...
I20260811 18:59:11.743755 31900 ts_manager.cc:194] Registered new tserver with Master: db36775e0a434ab1acd7f4fd4f9cb1a9 (127.27.180.129:42995)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.180.130@KRBTEST.COM" created.
I20260811 18:59:11.744998 31900 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.180.129@KRBTEST.COM'} at 127.27.180.129:33337
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.180.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.130.keytab.
Entry for principal kudu/127.27.180.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.130.keytab.
I20260811 18:59:11.759877 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474496157106-28370-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474496157106-28370-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474496157106-28370-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474496157106-28370-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.180.130:0
--local_ip_for_outbound_sockets=127.27.180.130
--webserver_interface=127.27.180.130
--webserver_port=0
--tserver_master_addrs=127.27.180.190:33077
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.130.keytab
--principal=kudu/127.27.180.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:33269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:59:11.869536 32277 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:59:11.869769 32277 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:59:11.869827 32277 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:59:11.873821 32277 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:59:11.873872 32277 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:59:11.873946 32277 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.180.130
I20260811 18:59:11.879016 32277 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:33269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474496157106-28370-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474496157106-28370-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.130.keytab
--principal=kudu/127.27.180.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.180.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474496157106-28370-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.180.130
--webserver_port=0
--tserver_master_addrs=127.27.180.190:33077
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32277
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.180.130
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474496157106-28370-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:59:11.879956 32277 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:59:11.880940 32277 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:59:11.889986 32286 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:11.889986 32292 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:11.889990 32287 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:59:11.890259 32277 server_base.cc:1034] running on GCE node
I20260811 18:59:11.890619 32277 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:59:11.891182 32277 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:59:11.892351 32277 hybrid_clock.cc:648] HybridClock initialized: now 1786474751892331 us; error 26 us; skew 500 ppm
Aug 11 18:59:11 dist-test-slave-2j7r krb5kdc[31804](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474751, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:59:11.894929 32277 init.cc:377] Logged in from keytab as kudu/127.27.180.130@KRBTEST.COM (short username kudu)
I20260811 18:59:11.895777 32277 webserver.cc:460] Webserver started at http://127.27.180.130:35737/ using document root <none> and password file <none>
I20260811 18:59:11.896279 32277 fs_manager.cc:359] Metadata directory not provided
I20260811 18:59:11.896342 32277 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:59:11.896546 32277 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:59:11.898504 32277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474496157106-28370-0/minicluster-data/ts-1/data/instance:
uuid: "9a972fc424c8464d80e42b862b6bb63e"
format_stamp: "Formatted at 2026-08-11 18:59:11 on dist-test-slave-2j7r"
server_key: "324c11d75aaae69520a1167b3b53bc39"
server_key_iv: "a9f0671247b79594f8629b83d4b3f427"
server_key_version: "encryptionkey@0"
I20260811 18:59:11.899029 32277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474496157106-28370-0/minicluster-data/ts-1/wal/instance:
uuid: "9a972fc424c8464d80e42b862b6bb63e"
format_stamp: "Formatted at 2026-08-11 18:59:11 on dist-test-slave-2j7r"
server_key: "324c11d75aaae69520a1167b3b53bc39"
server_key_iv: "a9f0671247b79594f8629b83d4b3f427"
server_key_version: "encryptionkey@0"
I20260811 18:59:11.902520 32277 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 18:59:11.904807 32302 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:11.906026 32277 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 18:59:11.906131 32277 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474496157106-28370-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474496157106-28370-0/minicluster-data/ts-1/wal
uuid: "9a972fc424c8464d80e42b862b6bb63e"
format_stamp: "Formatted at 2026-08-11 18:59:11 on dist-test-slave-2j7r"
server_key: "324c11d75aaae69520a1167b3b53bc39"
server_key_iv: "a9f0671247b79594f8629b83d4b3f427"
server_key_version: "encryptionkey@0"
I20260811 18:59:11.906219 32277 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474496157106-28370-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474496157106-28370-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474496157106-28370-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:59:11.932575 32277 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:59:11.935384 32277 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:59:11.935556 32277 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:59:11.936148 32277 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:59:11.936964 32277 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:59:11.937012 32277 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:11.937064 32277 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:59:11.937150 32277 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:11.945712 32277 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.130:42733
I20260811 18:59:11.945750 32501 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.130:42733 every 8 connection(s)
I20260811 18:59:11.946630 32277 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474496157106-28370-0/minicluster-data/ts-1/data/info.pb
Aug 11 18:59:11 dist-test-slave-2j7r krb5kdc[31804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474751, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.130@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM
I20260811 18:59:11.956459 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 32277
I20260811 18:59:11.956604 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474496157106-28370-0/minicluster-data/ts-1/wal/instance
I20260811 18:59:11.956889 28370 external_mini_cluster.cc:1428] Setting key 18663bfd7080ccbf0a8b3c5111799613
I20260811 18:59:11.959900 31972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:11.948401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.130:47823 (local address 127.27.180.190:33077)
0811 18:59:11.948544 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:11.948547 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:11.949357 (+ 810us) server_negotiation.cc:407] Connection header received
0811 18:59:11.950652 (+ 1295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:11.950655 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:11.950712 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:11.950798 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:11.951931 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:11.952460 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:11.953626 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:11.953860 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:11.953952 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:11.956233 (+ 2281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:11.956237 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:11.956240 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:59:11.956266 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:11.957596 (+ 1330us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:11.958254 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:11.958259 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:11.958261 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:11.958372 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:11.958757 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:11.958761 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:11.958763 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:11.958965 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:11.959065 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:11.959578 (+ 513us) server_negotiation.cc:299] Negotiation successful
0811 18:59:11.959718 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 18:59:11.960693 32504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:11.948726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33077 (local address 127.27.180.130:47823)
0811 18:59:11.949184 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:11.949254 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0811 18:59:11.950443 (+ 1189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:59:11.950977 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:59:11.950984 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:59:11.951331 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:59:11.951782 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:11.951796 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:11.952582 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:11.952585 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:11.953506 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:11.953516 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:11.954015 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:11.954020 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:11.954175 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:11.954662 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:59:11.954678 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:59:11.956068 (+ 1390us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:59:11.957725 (+ 1657us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:59:11.957731 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:59:11.957745 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:59:11.958080 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:59:11.958479 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:59:11.958483 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:59:11.958485 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:59:11.958648 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:59:11.959076 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:59:11.959083 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:59:11.959326 (+ 243us) client_negotiation.cc:769] Sending connection context
0811 18:59:11.959504 (+ 178us) client_negotiation.cc:240] Negotiation successful
0811 18:59:11.959856 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":111,"threads_started":1}
I20260811 18:59:11.961966 32502 heartbeater.cc:347] Connected to a master server at 127.27.180.190:33077
I20260811 18:59:11.962220 32502 heartbeater.cc:464] Registering TS with master...
I20260811 18:59:11.962698 32502 heartbeater.cc:511] Master 127.27.180.190:33077 requested a full tablet report, sending...
I20260811 18:59:11.963732 31900 ts_manager.cc:194] Registered new tserver with Master: 9a972fc424c8464d80e42b862b6bb63e (127.27.180.130:42733)
I20260811 18:59:11.964282 31900 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.180.130@KRBTEST.COM'} at 127.27.180.130:47823
WARNING: no policy specified for kudu/127.27.180.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.180.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.180.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.131.keytab.
Entry for principal kudu/127.27.180.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.131.keytab.
I20260811 18:59:11.985431 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474496157106-28370-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474496157106-28370-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474496157106-28370-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474496157106-28370-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.180.131:0
--local_ip_for_outbound_sockets=127.27.180.131
--webserver_interface=127.27.180.131
--webserver_port=0
--tserver_master_addrs=127.27.180.190:33077
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.131.keytab
--principal=kudu/127.27.180.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:33269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:59:12.098198 32510 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:59:12.098444 32510 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:59:12.098489 32510 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:59:12.102475 32510 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:59:12.102535 32510 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:59:12.102607 32510 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.180.131
I20260811 18:59:12.107882 32510 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:33269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474496157106-28370-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474496157106-28370-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.131.keytab
--principal=kudu/127.27.180.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.180.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474496157106-28370-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.180.131
--webserver_port=0
--tserver_master_addrs=127.27.180.190:33077
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32510
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.180.131
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474496157106-28370-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:59:12.108915 32510 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:59:12.109954 32510 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:59:12.119222 32521 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:12.119340 32515 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:12.119342 32517 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:59:12.119390 32510 server_base.cc:1034] running on GCE node
I20260811 18:59:12.119904 32510 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:59:12.120507 32510 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:59:12.121675 32510 hybrid_clock.cc:648] HybridClock initialized: now 1786474752121652 us; error 31 us; skew 500 ppm
Aug 11 18:59:12 dist-test-slave-2j7r krb5kdc[31804](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474752, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:59:12.124380 32510 init.cc:377] Logged in from keytab as kudu/127.27.180.131@KRBTEST.COM (short username kudu)
I20260811 18:59:12.125312 32510 webserver.cc:460] Webserver started at http://127.27.180.131:41993/ using document root <none> and password file <none>
I20260811 18:59:12.125839 32510 fs_manager.cc:359] Metadata directory not provided
I20260811 18:59:12.125912 32510 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:59:12.126142 32510 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:59:12.128158 32510 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474496157106-28370-0/minicluster-data/ts-2/data/instance:
uuid: "2cd4ed2ce4644ec4ade9688df8951bf3"
format_stamp: "Formatted at 2026-08-11 18:59:12 on dist-test-slave-2j7r"
server_key: "10b0212763cf3618efcc937dfa56c392"
server_key_iv: "9ed1ce68f15f70ddf52c313ffbc0f111"
server_key_version: "encryptionkey@0"
I20260811 18:59:12.128729 32510 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474496157106-28370-0/minicluster-data/ts-2/wal/instance:
uuid: "2cd4ed2ce4644ec4ade9688df8951bf3"
format_stamp: "Formatted at 2026-08-11 18:59:12 on dist-test-slave-2j7r"
server_key: "10b0212763cf3618efcc937dfa56c392"
server_key_iv: "9ed1ce68f15f70ddf52c313ffbc0f111"
server_key_version: "encryptionkey@0"
I20260811 18:59:12.132375 32510 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260811 18:59:12.134764 32533 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:12.135922 32510 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 18:59:12.136042 32510 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474496157106-28370-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474496157106-28370-0/minicluster-data/ts-2/wal
uuid: "2cd4ed2ce4644ec4ade9688df8951bf3"
format_stamp: "Formatted at 2026-08-11 18:59:12 on dist-test-slave-2j7r"
server_key: "10b0212763cf3618efcc937dfa56c392"
server_key_iv: "9ed1ce68f15f70ddf52c313ffbc0f111"
server_key_version: "encryptionkey@0"
I20260811 18:59:12.136139 32510 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474496157106-28370-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474496157106-28370-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474496157106-28370-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:59:12.155562 32510 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:59:12.158234 32510 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:59:12.158399 32510 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:59:12.158955 32510 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:59:12.159770 32510 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:59:12.159873 32510 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:12.159976 32510 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:59:12.160007 32510 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:12.168218 32510 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.131:38689
I20260811 18:59:12.168242 32738 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.131:38689 every 8 connection(s)
I20260811 18:59:12.169067 32510 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474496157106-28370-0/minicluster-data/ts-2/data/info.pb
I20260811 18:59:12.171367 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 32510
I20260811 18:59:12.171505 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474496157106-28370-0/minicluster-data/ts-2/wal/instance
I20260811 18:59:12.171836 28370 external_mini_cluster.cc:1428] Setting key 3a9a0b0d49e51c32c5e6b957d07ce9b8
Aug 11 18:59:12 dist-test-slave-2j7r krb5kdc[31804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474752, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.131@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM
I20260811 18:59:12.182082 31972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:12.170812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.131:44523 (local address 127.27.180.190:33077)
0811 18:59:12.170955 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:12.170960 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:12.171948 (+ 988us) server_negotiation.cc:407] Connection header received
0811 18:59:12.173461 (+ 1513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:12.173465 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:12.173515 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:12.173597 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:12.174754 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:12.175320 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:12.176380 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:12.176597 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:12.176676 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:12.178942 (+ 2266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:12.178945 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:12.178947 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:59:12.178973 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:12.180205 (+ 1232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:12.180638 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:12.180640 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:12.180641 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:12.180713 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:12.181010 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:12.181014 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:12.181015 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:12.181286 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:12.181385 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:12.181854 (+ 469us) server_negotiation.cc:299] Negotiation successful
0811 18:59:12.181963 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 18:59:12.182799 32743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:12.171246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:33077 (local address 127.27.180.131:44523)
0811 18:59:12.171781 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:12.171839 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 18:59:12.173263 (+ 1424us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:59:12.173740 (+ 477us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:59:12.173746 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:59:12.174119 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:59:12.174602 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:12.174616 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:12.175415 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:12.175417 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:12.176265 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:12.176274 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:12.176709 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:12.176711 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:12.176842 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:12.177404 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:59:12.177420 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:59:12.178780 (+ 1360us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:59:12.180303 (+ 1523us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:59:12.180307 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:59:12.180318 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:59:12.180562 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:59:12.180777 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:59:12.180779 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:59:12.180780 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:59:12.180901 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:59:12.181387 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:59:12.181391 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:59:12.181607 (+ 216us) client_negotiation.cc:769] Sending connection context
0811 18:59:12.181757 (+ 150us) client_negotiation.cc:240] Negotiation successful
0811 18:59:12.181992 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":115,"threads_started":1}
I20260811 18:59:12.183817 32740 heartbeater.cc:347] Connected to a master server at 127.27.180.190:33077
I20260811 18:59:12.184026 32740 heartbeater.cc:464] Registering TS with master...
I20260811 18:59:12.184508 32740 heartbeater.cc:511] Master 127.27.180.190:33077 requested a full tablet report, sending...
I20260811 18:59:12.185539 31897 ts_manager.cc:194] Registered new tserver with Master: 2cd4ed2ce4644ec4ade9688df8951bf3 (127.27.180.131:38689)
I20260811 18:59:12.186129 31897 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.27.180.131@KRBTEST.COM'} at 127.27.180.131:44523
I20260811 18:59:12.197058 28370 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 18:59:12.205333 31972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:12.199025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55150 (local address 127.27.180.190:33077)
0811 18:59:12.199146 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:12.199151 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:12.199309 (+ 158us) server_negotiation.cc:407] Connection header received
0811 18:59:12.199385 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:12.199387 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:12.199423 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:12.199482 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:12.200082 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:12.200588 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:12.201384 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:12.201588 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:12.201661 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:12.202799 (+ 1138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:12.202802 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:12.202804 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:59:12.202825 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:12.203913 (+ 1088us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:12.204324 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:12.204326 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:12.204328 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:12.204404 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:12.204587 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:12.204588 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:12.204589 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:12.204774 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:12.204869 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:12.205015 (+ 146us) server_negotiation.cc:299] Negotiation successful
0811 18:59:12.205158 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 18:59:12.207989 31897 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55150:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:59:12.211017 31897 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:59:12.224761 32765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:12.219151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.130:42733 (local address 127.0.0.1:60638)
0811 18:59:12.220233 (+ 1082us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:12.220246 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:59:12.220367 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:59:12.220905 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:59:12.220911 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:59:12.220945 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:59:12.221213 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:12.221221 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:12.222233 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:12.222236 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:12.223374 (+ 1138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:12.223383 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:12.224179 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:12.224182 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:12.224295 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:12.224372 (+ 77us) client_negotiation.cc:769] Sending connection context
0811 18:59:12.224481 (+ 109us) client_negotiation.cc:240] Negotiation successful
0811 18:59:12.224605 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1028,"mutex_wait_us":82,"spinlock_wait_cycles":9472,"thread_start_us":89,"threads_started":1}
I20260811 18:59:12.224778 302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:12.220035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60638 (local address 127.27.180.130:42733)
0811 18:59:12.220446 (+ 411us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:12.220459 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:12.220483 (+ 24us) server_negotiation.cc:407] Connection header received
0811 18:59:12.220553 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:12.220557 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:12.220747 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:12.220902 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:59:12.221343 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:12.222102 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:12.223490 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:12.224059 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:12.224176 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:12.224316 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:12.224527 (+ 211us) server_negotiation.cc:299] Negotiation successful
0811 18:59:12.224633 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326,"thread_start_us":116,"threads_started":1}
I20260811 18:59:12.225419 32764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:12.219067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.129:42995 (local address 127.0.0.1:34038)
0811 18:59:12.219498 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:12.219550 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 18:59:12.219674 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:59:12.220915 (+ 1241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:59:12.220917 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:59:12.220947 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:59:12.221213 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:12.221221 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:12.222780 (+ 1559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:12.222783 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:12.223943 (+ 1160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:12.223952 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:12.225002 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:12.225005 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:12.225112 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:12.225134 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 18:59:12.225210 (+ 76us) client_negotiation.cc:240] Negotiation successful
0811 18:59:12.225289 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"mutex_wait_us":99,"spinlock_wait_cycles":8448,"thread_start_us":78,"threads_started":1}
I20260811 18:59:12.225446 32766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:12.219218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34038 (local address 127.27.180.129:42995)
0811 18:59:12.220006 (+ 788us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:12.220044 (+ 38us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:12.220070 (+ 26us) server_negotiation.cc:407] Connection header received
0811 18:59:12.220569 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:12.220576 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:12.220783 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:12.221358 (+ 575us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:59:12.221506 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:12.222636 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:12.224057 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:12.224863 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:12.225011 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:12.225159 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:12.225237 (+ 78us) server_negotiation.cc:299] Negotiation successful
0811 18:59:12.225325 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":674,"thread_start_us":109,"threads_started":1}
I20260811 18:59:12.226318 300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:12.219833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.131:38689 (local address 127.0.0.1:38770)
0811 18:59:12.220548 (+ 715us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:12.220560 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 18:59:12.220646 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:59:12.221202 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:59:12.221204 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:59:12.221220 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:59:12.221297 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:12.221302 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:12.222717 (+ 1415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:12.222721 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:12.224330 (+ 1609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:12.224339 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:12.225685 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:12.225688 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:12.226036 (+ 348us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:12.226062 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 18:59:12.226134 (+ 72us) client_negotiation.cc:240] Negotiation successful
0811 18:59:12.226199 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":657,"mutex_wait_us":62,"thread_start_us":91,"threads_started":1}
I20260811 18:59:12.226728 301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:12.220005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38770 (local address 127.27.180.131:38689)
0811 18:59:12.220447 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:12.220459 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:12.220646 (+ 187us) server_negotiation.cc:407] Connection header received
0811 18:59:12.220737 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:12.220742 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:12.220910 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:12.221043 (+ 133us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:59:12.221397 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:12.222566 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:12.224430 (+ 1864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:12.225393 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:12.225506 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:12.225622 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:12.226498 (+ 876us) server_negotiation.cc:299] Negotiation successful
0811 18:59:12.226599 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":144,"threads_started":1}
I20260811 18:59:12.228300 32392 tablet_service.cc:1467] Processing CreateTablet for tablet 4607e7e51cd549ca96f38ddf8ad4a26d (DEFAULT_TABLE table=test-table [id=2394153a860b49fcb7d45a97b4bdd863]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:59:12.229295 32392 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4607e7e51cd549ca96f38ddf8ad4a26d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:59:12.229022 32626 tablet_service.cc:1467] Processing CreateTablet for tablet 4607e7e51cd549ca96f38ddf8ad4a26d (DEFAULT_TABLE table=test-table [id=2394153a860b49fcb7d45a97b4bdd863]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:59:12.230098 32626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4607e7e51cd549ca96f38ddf8ad4a26d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:59:12.231091 32142 tablet_service.cc:1467] Processing CreateTablet for tablet 4607e7e51cd549ca96f38ddf8ad4a26d (DEFAULT_TABLE table=test-table [id=2394153a860b49fcb7d45a97b4bdd863]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:59:12.232048 32142 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4607e7e51cd549ca96f38ddf8ad4a26d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:59:12.235512 306 tablet_bootstrap.cc:492] T 4607e7e51cd549ca96f38ddf8ad4a26d P 9a972fc424c8464d80e42b862b6bb63e: Bootstrap starting.
I20260811 18:59:12.236922 307 tablet_bootstrap.cc:492] T 4607e7e51cd549ca96f38ddf8ad4a26d P 2cd4ed2ce4644ec4ade9688df8951bf3: Bootstrap starting.
I20260811 18:59:12.237913 306 tablet_bootstrap.cc:654] T 4607e7e51cd549ca96f38ddf8ad4a26d P 9a972fc424c8464d80e42b862b6bb63e: Neither blocks nor log segments found. Creating new log.
I20260811 18:59:12.238505 311 tablet_bootstrap.cc:492] T 4607e7e51cd549ca96f38ddf8ad4a26d P db36775e0a434ab1acd7f4fd4f9cb1a9: Bootstrap starting.
I20260811 18:59:12.239094 307 tablet_bootstrap.cc:654] T 4607e7e51cd549ca96f38ddf8ad4a26d P 2cd4ed2ce4644ec4ade9688df8951bf3: Neither blocks nor log segments found. Creating new log.
I20260811 18:59:12.239168 306 log.cc:824] T 4607e7e51cd549ca96f38ddf8ad4a26d P 9a972fc424c8464d80e42b862b6bb63e: Log is configured to *not* fsync() on all Append() calls
I20260811 18:59:12.240325 307 log.cc:824] T 4607e7e51cd549ca96f38ddf8ad4a26d P 2cd4ed2ce4644ec4ade9688df8951bf3: Log is configured to *not* fsync() on all Append() calls
I20260811 18:59:12.240548 311 tablet_bootstrap.cc:654] T 4607e7e51cd549ca96f38ddf8ad4a26d P db36775e0a434ab1acd7f4fd4f9cb1a9: Neither blocks nor log segments found. Creating new log.
I20260811 18:59:12.241495 306 tablet_bootstrap.cc:492] T 4607e7e51cd549ca96f38ddf8ad4a26d P 9a972fc424c8464d80e42b862b6bb63e: No bootstrap required, opened a new log
I20260811 18:59:12.241691 306 ts_tablet_manager.cc:1397] T 4607e7e51cd549ca96f38ddf8ad4a26d P 9a972fc424c8464d80e42b862b6bb63e: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 18:59:12.241724 311 log.cc:824] T 4607e7e51cd549ca96f38ddf8ad4a26d P db36775e0a434ab1acd7f4fd4f9cb1a9: Log is configured to *not* fsync() on all Append() calls
I20260811 18:59:12.242156 307 tablet_bootstrap.cc:492] T 4607e7e51cd549ca96f38ddf8ad4a26d P 2cd4ed2ce4644ec4ade9688df8951bf3: No bootstrap required, opened a new log
I20260811 18:59:12.242349 307 ts_tablet_manager.cc:1397] T 4607e7e51cd549ca96f38ddf8ad4a26d P 2cd4ed2ce4644ec4ade9688df8951bf3: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 18:59:12.243602 311 tablet_bootstrap.cc:492] T 4607e7e51cd549ca96f38ddf8ad4a26d P db36775e0a434ab1acd7f4fd4f9cb1a9: No bootstrap required, opened a new log
I20260811 18:59:12.243772 311 ts_tablet_manager.cc:1397] T 4607e7e51cd549ca96f38ddf8ad4a26d P db36775e0a434ab1acd7f4fd4f9cb1a9: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260811 18:59:12.244642 306 raft_consensus.cc:348] T 4607e7e51cd549ca96f38ddf8ad4a26d P 9a972fc424c8464d80e42b862b6bb63e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db36775e0a434ab1acd7f4fd4f9cb1a9" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 42995 } } peers { permanent_uuid: "9a972fc424c8464d80e42b862b6bb63e" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 42733 } } peers { permanent_uuid: "2cd4ed2ce4644ec4ade9688df8951bf3" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 38689 } }
I20260811 18:59:12.244853 306 raft_consensus.cc:374] T 4607e7e51cd549ca96f38ddf8ad4a26d P 9a972fc424c8464d80e42b862b6bb63e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:59:12.244925 306 raft_consensus.cc:729] T 4607e7e51cd549ca96f38ddf8ad4a26d P 9a972fc424c8464d80e42b862b6bb63e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a972fc424c8464d80e42b862b6bb63e, State: Initialized, Role: FOLLOWER
I20260811 18:59:12.245360 306 consensus_queue.cc:260] T 4607e7e51cd549ca96f38ddf8ad4a26d P 9a972fc424c8464d80e42b862b6bb63e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db36775e0a434ab1acd7f4fd4f9cb1a9" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 42995 } } peers { permanent_uuid: "9a972fc424c8464d80e42b862b6bb63e" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 42733 } } peers { permanent_uuid: "2cd4ed2ce4644ec4ade9688df8951bf3" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 38689 } }
I20260811 18:59:12.245988 32502 heartbeater.cc:503] Master 127.27.180.190:33077 was elected leader, sending a full tablet report...
I20260811 18:59:12.246054 307 raft_consensus.cc:348] T 4607e7e51cd549ca96f38ddf8ad4a26d P 2cd4ed2ce4644ec4ade9688df8951bf3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db36775e0a434ab1acd7f4fd4f9cb1a9" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 42995 } } peers { permanent_uuid: "9a972fc424c8464d80e42b862b6bb63e" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 42733 } } peers { permanent_uuid: "2cd4ed2ce4644ec4ade9688df8951bf3" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 38689 } }
I20260811 18:59:12.246284 307 raft_consensus.cc:374] T 4607e7e51cd549ca96f38ddf8ad4a26d P 2cd4ed2ce4644ec4ade9688df8951bf3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:59:12.246330 307 raft_consensus.cc:729] T 4607e7e51cd549ca96f38ddf8ad4a26d P 2cd4ed2ce4644ec4ade9688df8951bf3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cd4ed2ce4644ec4ade9688df8951bf3, State: Initialized, Role: FOLLOWER
I20260811 18:59:12.246332 306 ts_tablet_manager.cc:1428] T 4607e7e51cd549ca96f38ddf8ad4a26d P 9a972fc424c8464d80e42b862b6bb63e: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260811 18:59:12.246758 307 consensus_queue.cc:260] T 4607e7e51cd549ca96f38ddf8ad4a26d P 2cd4ed2ce4644ec4ade9688df8951bf3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db36775e0a434ab1acd7f4fd4f9cb1a9" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 42995 } } peers { permanent_uuid: "9a972fc424c8464d80e42b862b6bb63e" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 42733 } } peers { permanent_uuid: "2cd4ed2ce4644ec4ade9688df8951bf3" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 38689 } }
I20260811 18:59:12.247745 307 ts_tablet_manager.cc:1428] T 4607e7e51cd549ca96f38ddf8ad4a26d P 2cd4ed2ce4644ec4ade9688df8951bf3: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260811 18:59:12.247592 311 raft_consensus.cc:348] T 4607e7e51cd549ca96f38ddf8ad4a26d P db36775e0a434ab1acd7f4fd4f9cb1a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db36775e0a434ab1acd7f4fd4f9cb1a9" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 42995 } } peers { permanent_uuid: "9a972fc424c8464d80e42b862b6bb63e" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 42733 } } peers { permanent_uuid: "2cd4ed2ce4644ec4ade9688df8951bf3" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 38689 } }
I20260811 18:59:12.247836 311 raft_consensus.cc:374] T 4607e7e51cd549ca96f38ddf8ad4a26d P db36775e0a434ab1acd7f4fd4f9cb1a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:59:12.247898 311 raft_consensus.cc:729] T 4607e7e51cd549ca96f38ddf8ad4a26d P db36775e0a434ab1acd7f4fd4f9cb1a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db36775e0a434ab1acd7f4fd4f9cb1a9, State: Initialized, Role: FOLLOWER
I20260811 18:59:12.248078 32740 heartbeater.cc:503] Master 127.27.180.190:33077 was elected leader, sending a full tablet report...
I20260811 18:59:12.248389 311 consensus_queue.cc:260] T 4607e7e51cd549ca96f38ddf8ad4a26d P db36775e0a434ab1acd7f4fd4f9cb1a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db36775e0a434ab1acd7f4fd4f9cb1a9" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 42995 } } peers { permanent_uuid: "9a972fc424c8464d80e42b862b6bb63e" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 42733 } } peers { permanent_uuid: "2cd4ed2ce4644ec4ade9688df8951bf3" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 38689 } }
I20260811 18:59:12.249284 32270 heartbeater.cc:503] Master 127.27.180.190:33077 was elected leader, sending a full tablet report...
I20260811 18:59:12.249316 311 ts_tablet_manager.cc:1428] T 4607e7e51cd549ca96f38ddf8ad4a26d P db36775e0a434ab1acd7f4fd4f9cb1a9: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20260811 18:59:12.285784 321 raft_consensus.cc:482] T 4607e7e51cd549ca96f38ddf8ad4a26d P 9a972fc424c8464d80e42b862b6bb63e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:59:12.286001 321 raft_consensus.cc:504] T 4607e7e51cd549ca96f38ddf8ad4a26d P 9a972fc424c8464d80e42b862b6bb63e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db36775e0a434ab1acd7f4fd4f9cb1a9" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 42995 } } peers { permanent_uuid: "9a972fc424c8464d80e42b862b6bb63e" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 42733 } } peers { permanent_uuid: "2cd4ed2ce4644ec4ade9688df8951bf3" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 38689 } }
I20260811 18:59:12.287130 321 leader_election.cc:288] T 4607e7e51cd549ca96f38ddf8ad4a26d P 9a972fc424c8464d80e42b862b6bb63e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers db36775e0a434ab1acd7f4fd4f9cb1a9 (127.27.180.129:42995), 2cd4ed2ce4644ec4ade9688df8951bf3 (127.27.180.131:38689)
I20260811 18:59:12.291412 32504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:12.287405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.131:38689 (local address 127.27.180.130:45401)
0811 18:59:12.287531 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:12.287548 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 18:59:12.287649 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:59:12.287909 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:59:12.287912 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:59:12.287938 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:59:12.288064 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:12.288071 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:12.289015 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:12.289018 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:12.290172 (+ 1154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:12.290183 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:12.291028 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:12.291031 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:12.291114 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:12.291136 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 18:59:12.291193 (+ 57us) client_negotiation.cc:240] Negotiation successful
0811 18:59:12.291249 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260811 18:59:12.291450 301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:12.287505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.130:45401 (local address 127.27.180.131:38689)
0811 18:59:12.287648 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:12.287654 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:12.287681 (+ 27us) server_negotiation.cc:407] Connection header received
0811 18:59:12.287740 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:12.287743 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:12.287799 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:12.287885 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:59:12.288212 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:12.288899 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:12.290339 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:12.290906 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:12.291045 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:12.291107 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:12.291231 (+ 124us) server_negotiation.cc:299] Negotiation successful
0811 18:59:12.291312 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 18:59:12.291774 32766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:12.287505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.130:49317 (local address 127.27.180.129:42995)
0811 18:59:12.287618 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:12.287622 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:12.288232 (+ 610us) server_negotiation.cc:407] Connection header received
0811 18:59:12.288299 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:12.288302 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:12.288356 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:12.288439 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:59:12.288686 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:12.289338 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:12.290603 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:12.291221 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:12.291352 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:12.291405 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:12.291589 (+ 184us) server_negotiation.cc:299] Negotiation successful
0811 18:59:12.291674 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 18:59:12.292121 331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:12.287405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.129:42995 (local address 127.27.180.130:49317)
0811 18:59:12.288102 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:12.288115 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:59:12.288207 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:59:12.288471 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:59:12.288473 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:59:12.288489 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:59:12.288589 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:12.288594 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:12.289458 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:12.289461 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:12.290480 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:12.290489 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:12.291347 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:12.291351 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:12.291438 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:12.291461 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 18:59:12.291918 (+ 457us) client_negotiation.cc:240] Negotiation successful
0811 18:59:12.291995 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":633,"mutex_wait_us":318,"thread_start_us":144,"threads_started":1}
I20260811 18:59:12.292140 32665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4607e7e51cd549ca96f38ddf8ad4a26d" candidate_uuid: "9a972fc424c8464d80e42b862b6bb63e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cd4ed2ce4644ec4ade9688df8951bf3" is_pre_election: true
I20260811 18:59:12.292457 32665 raft_consensus.cc:2454] T 4607e7e51cd549ca96f38ddf8ad4a26d P 2cd4ed2ce4644ec4ade9688df8951bf3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a972fc424c8464d80e42b862b6bb63e in term 0.
I20260811 18:59:12.292766 32186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4607e7e51cd549ca96f38ddf8ad4a26d" candidate_uuid: "9a972fc424c8464d80e42b862b6bb63e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db36775e0a434ab1acd7f4fd4f9cb1a9" is_pre_election: true
I20260811 18:59:12.292906 32304 leader_election.cc:302] T 4607e7e51cd549ca96f38ddf8ad4a26d P 9a972fc424c8464d80e42b862b6bb63e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cd4ed2ce4644ec4ade9688df8951bf3, 9a972fc424c8464d80e42b862b6bb63e; no voters:
I20260811 18:59:12.293112 32186 raft_consensus.cc:2454] T 4607e7e51cd549ca96f38ddf8ad4a26d P db36775e0a434ab1acd7f4fd4f9cb1a9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a972fc424c8464d80e42b862b6bb63e in term 0.
I20260811 18:59:12.293224 321 raft_consensus.cc:2781] T 4607e7e51cd549ca96f38ddf8ad4a26d P 9a972fc424c8464d80e42b862b6bb63e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:59:12.293316 321 raft_consensus.cc:482] T 4607e7e51cd549ca96f38ddf8ad4a26d P 9a972fc424c8464d80e42b862b6bb63e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:59:12.293363 321 raft_consensus.cc:3037] T 4607e7e51cd549ca96f38ddf8ad4a26d P 9a972fc424c8464d80e42b862b6bb63e [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:59:12.294512 321 raft_consensus.cc:504] T 4607e7e51cd549ca96f38ddf8ad4a26d P 9a972fc424c8464d80e42b862b6bb63e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db36775e0a434ab1acd7f4fd4f9cb1a9" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 42995 } } peers { permanent_uuid: "9a972fc424c8464d80e42b862b6bb63e" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 42733 } } peers { permanent_uuid: "2cd4ed2ce4644ec4ade9688df8951bf3" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 38689 } }
I20260811 18:59:12.294862 321 leader_election.cc:288] T 4607e7e51cd549ca96f38ddf8ad4a26d P 9a972fc424c8464d80e42b862b6bb63e [CANDIDATE]: Term 1 election: Requested vote from peers db36775e0a434ab1acd7f4fd4f9cb1a9 (127.27.180.129:42995), 2cd4ed2ce4644ec4ade9688df8951bf3 (127.27.180.131:38689)
I20260811 18:59:12.295226 32186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4607e7e51cd549ca96f38ddf8ad4a26d" candidate_uuid: "9a972fc424c8464d80e42b862b6bb63e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db36775e0a434ab1acd7f4fd4f9cb1a9"
I20260811 18:59:12.295356 32186 raft_consensus.cc:3037] T 4607e7e51cd549ca96f38ddf8ad4a26d P db36775e0a434ab1acd7f4fd4f9cb1a9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:59:12.296376 32665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4607e7e51cd549ca96f38ddf8ad4a26d" candidate_uuid: "9a972fc424c8464d80e42b862b6bb63e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cd4ed2ce4644ec4ade9688df8951bf3"
I20260811 18:59:12.296437 32186 raft_consensus.cc:2454] T 4607e7e51cd549ca96f38ddf8ad4a26d P db36775e0a434ab1acd7f4fd4f9cb1a9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a972fc424c8464d80e42b862b6bb63e in term 1.
I20260811 18:59:12.296492 32665 raft_consensus.cc:3037] T 4607e7e51cd549ca96f38ddf8ad4a26d P 2cd4ed2ce4644ec4ade9688df8951bf3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:59:12.296730 32307 leader_election.cc:302] T 4607e7e51cd549ca96f38ddf8ad4a26d P 9a972fc424c8464d80e42b862b6bb63e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9a972fc424c8464d80e42b862b6bb63e, db36775e0a434ab1acd7f4fd4f9cb1a9; no voters:
I20260811 18:59:12.296948 321 raft_consensus.cc:2781] T 4607e7e51cd549ca96f38ddf8ad4a26d P 9a972fc424c8464d80e42b862b6bb63e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:59:12.297189 321 raft_consensus.cc:686] T 4607e7e51cd549ca96f38ddf8ad4a26d P 9a972fc424c8464d80e42b862b6bb63e [term 1 LEADER]: Becoming Leader. State: Replica: 9a972fc424c8464d80e42b862b6bb63e, State: Running, Role: LEADER
I20260811 18:59:12.297552 321 consensus_queue.cc:237] T 4607e7e51cd549ca96f38ddf8ad4a26d P 9a972fc424c8464d80e42b862b6bb63e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db36775e0a434ab1acd7f4fd4f9cb1a9" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 42995 } } peers { permanent_uuid: "9a972fc424c8464d80e42b862b6bb63e" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 42733 } } peers { permanent_uuid: "2cd4ed2ce4644ec4ade9688df8951bf3" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 38689 } }
I20260811 18:59:12.297778 32665 raft_consensus.cc:2454] T 4607e7e51cd549ca96f38ddf8ad4a26d P 2cd4ed2ce4644ec4ade9688df8951bf3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a972fc424c8464d80e42b862b6bb63e in term 1.
I20260811 18:59:12.301374 31897 catalog_manager.cc:5495] T 4607e7e51cd549ca96f38ddf8ad4a26d P 9a972fc424c8464d80e42b862b6bb63e reported cstate change: term changed from 0 to 1, leader changed from <none> to 9a972fc424c8464d80e42b862b6bb63e (127.27.180.130). New cstate: current_term: 1 leader_uuid: "9a972fc424c8464d80e42b862b6bb63e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db36775e0a434ab1acd7f4fd4f9cb1a9" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 42995 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a972fc424c8464d80e42b862b6bb63e" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 42733 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2cd4ed2ce4644ec4ade9688df8951bf3" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 38689 } health_report { overall_health: UNKNOWN } } }
I20260811 18:59:12.320369 302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:12.316966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60642 (local address 127.27.180.130:42733)
0811 18:59:12.317077 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:12.317114 (+ 37us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:12.317129 (+ 15us) server_negotiation.cc:407] Connection header received
0811 18:59:12.317290 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:12.317293 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:12.317348 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:12.317440 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:59:12.317686 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:12.318215 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:12.319032 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:12.319245 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:12.319331 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:12.319608 (+ 277us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:59:12.319991 (+ 383us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:59:12.320096 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:12.320167 (+ 71us) server_negotiation.cc:299] Negotiation successful
0811 18:59:12.320222 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 18:59:12.325978 32186 raft_consensus.cc:1261] T 4607e7e51cd549ca96f38ddf8ad4a26d P db36775e0a434ab1acd7f4fd4f9cb1a9 [term 1 FOLLOWER]: Refusing update from remote peer 9a972fc424c8464d80e42b862b6bb63e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:59:12.325978 32665 raft_consensus.cc:1261] T 4607e7e51cd549ca96f38ddf8ad4a26d P 2cd4ed2ce4644ec4ade9688df8951bf3 [term 1 FOLLOWER]: Refusing update from remote peer 9a972fc424c8464d80e42b862b6bb63e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:59:12.326673 321 consensus_queue.cc:1035] T 4607e7e51cd549ca96f38ddf8ad4a26d P 9a972fc424c8464d80e42b862b6bb63e [LEADER]: Connected to new peer: Peer: permanent_uuid: "2cd4ed2ce4644ec4ade9688df8951bf3" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 38689 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:59:12.326856 334 consensus_queue.cc:1035] T 4607e7e51cd549ca96f38ddf8ad4a26d P 9a972fc424c8464d80e42b862b6bb63e [LEADER]: Connected to new peer: Peer: permanent_uuid: "db36775e0a434ab1acd7f4fd4f9cb1a9" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 42995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:59:12.334949 338 mvcc.cc:205] Tried to move back new op lower bound from 7317400585519722496 to 7317400585414840320. Current Snapshot: MvccSnapshot[applied={T|T < 7317400585519722496}]
I20260811 18:59:12.336602 340 mvcc.cc:205] Tried to move back new op lower bound from 7317400585519722496 to 7317400585414840320. Current Snapshot: MvccSnapshot[applied={T|T < 7317400585519722496}]
I20260811 18:59:12.341897 31900 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55150:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:59:12.342037 31900 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55150:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:59:12.344352 31900 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f1d8d163358442eb9e9aa032d526a29c: Sending DeleteTablet for 3 replicas of tablet 4607e7e51cd549ca96f38ddf8ad4a26d
I20260811 18:59:12.345060 32142 tablet_service.cc:1514] Processing DeleteTablet for tablet 4607e7e51cd549ca96f38ddf8ad4a26d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:59:12 UTC) from {username='kudu', principal='kudu/127.27.180.190@KRBTEST.COM'} at 127.0.0.1:34038
I20260811 18:59:12.345109 32392 tablet_service.cc:1514] Processing DeleteTablet for tablet 4607e7e51cd549ca96f38ddf8ad4a26d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:59:12 UTC) from {username='kudu', principal='kudu/127.27.180.190@KRBTEST.COM'} at 127.0.0.1:60638
I20260811 18:59:12.345350 32626 tablet_service.cc:1514] Processing DeleteTablet for tablet 4607e7e51cd549ca96f38ddf8ad4a26d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:59:12 UTC) from {username='kudu', principal='kudu/127.27.180.190@KRBTEST.COM'} at 127.0.0.1:38770
I20260811 18:59:12.345763 356 tablet_replica.cc:330] T 4607e7e51cd549ca96f38ddf8ad4a26d P 2cd4ed2ce4644ec4ade9688df8951bf3: stopping tablet replica
I20260811 18:59:12.345916 353 tablet_replica.cc:330] T 4607e7e51cd549ca96f38ddf8ad4a26d P db36775e0a434ab1acd7f4fd4f9cb1a9: stopping tablet replica
I20260811 18:59:12.346138 356 raft_consensus.cc:2229] T 4607e7e51cd549ca96f38ddf8ad4a26d P 2cd4ed2ce4644ec4ade9688df8951bf3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:59:12.346213 353 raft_consensus.cc:2229] T 4607e7e51cd549ca96f38ddf8ad4a26d P db36775e0a434ab1acd7f4fd4f9cb1a9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:59:12.346501 353 raft_consensus.cc:2258] T 4607e7e51cd549ca96f38ddf8ad4a26d P db36775e0a434ab1acd7f4fd4f9cb1a9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:59:12.346501 356 raft_consensus.cc:2258] T 4607e7e51cd549ca96f38ddf8ad4a26d P 2cd4ed2ce4644ec4ade9688df8951bf3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:59:12.346505 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 32015
I20260811 18:59:12.354044 354 tablet_replica.cc:330] T 4607e7e51cd549ca96f38ddf8ad4a26d P 9a972fc424c8464d80e42b862b6bb63e: stopping tablet replica
I20260811 18:59:12.355455 356 ts_tablet_manager.cc:1905] T 4607e7e51cd549ca96f38ddf8ad4a26d P 2cd4ed2ce4644ec4ade9688df8951bf3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:59:12.359232 356 ts_tablet_manager.cc:1918] T 4607e7e51cd549ca96f38ddf8ad4a26d P 2cd4ed2ce4644ec4ade9688df8951bf3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:59:12.359403 356 log.cc:1196] T 4607e7e51cd549ca96f38ddf8ad4a26d P 2cd4ed2ce4644ec4ade9688df8951bf3: Deleting WAL directory at /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474496157106-28370-0/minicluster-data/ts-2/wal/wals/4607e7e51cd549ca96f38ddf8ad4a26d
I20260811 18:59:12.359756 356 ts_tablet_manager.cc:1939] T 4607e7e51cd549ca96f38ddf8ad4a26d P 2cd4ed2ce4644ec4ade9688df8951bf3: Deleting consensus metadata
I20260811 18:59:12.360662 354 raft_consensus.cc:2229] T 4607e7e51cd549ca96f38ddf8ad4a26d P 9a972fc424c8464d80e42b862b6bb63e [term 1 LEADER]: Raft consensus shutting down.
I20260811 18:59:12.361171 354 raft_consensus.cc:2258] T 4607e7e51cd549ca96f38ddf8ad4a26d P 9a972fc424c8464d80e42b862b6bb63e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:59:12.361171 31866 catalog_manager.cc:4841] TS 2cd4ed2ce4644ec4ade9688df8951bf3 (127.27.180.131:38689): tablet 4607e7e51cd549ca96f38ddf8ad4a26d (table test-table [id=2394153a860b49fcb7d45a97b4bdd863]) successfully deleted
I20260811 18:59:12.364019 354 ts_tablet_manager.cc:1905] T 4607e7e51cd549ca96f38ddf8ad4a26d P 9a972fc424c8464d80e42b862b6bb63e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:59:12.366778 354 ts_tablet_manager.cc:1918] T 4607e7e51cd549ca96f38ddf8ad4a26d P 9a972fc424c8464d80e42b862b6bb63e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:59:12.366876 354 log.cc:1196] T 4607e7e51cd549ca96f38ddf8ad4a26d P 9a972fc424c8464d80e42b862b6bb63e: Deleting WAL directory at /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786474496157106-28370-0/minicluster-data/ts-1/wal/wals/4607e7e51cd549ca96f38ddf8ad4a26d
I20260811 18:59:12.367197 354 ts_tablet_manager.cc:1939] T 4607e7e51cd549ca96f38ddf8ad4a26d P 9a972fc424c8464d80e42b862b6bb63e: Deleting consensus metadata
I20260811 18:59:12.368552 31865 catalog_manager.cc:4841] TS 9a972fc424c8464d80e42b862b6bb63e (127.27.180.130:42733): tablet 4607e7e51cd549ca96f38ddf8ad4a26d (table test-table [id=2394153a860b49fcb7d45a97b4bdd863]) successfully deleted
W20260811 18:59:12.375097 31871 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.27.180.129:42995 (error 108)
I20260811 18:59:12.375339 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 32277
I20260811 18:59:12.383713 32765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:12.383365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.129:42995 (local address 127.0.0.1:34040)
0811 18:59:12.383554 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:12.383623 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.129:42995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117}
W20260811 18:59:12.384073 31871 catalog_manager.cc:4584] TS db36775e0a434ab1acd7f4fd4f9cb1a9 (127.27.180.129:42995): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4607e7e51cd549ca96f38ddf8ad4a26d: Network error: Client connection negotiation failed: client connection to 127.27.180.129:42995: connect: Connection refused (error 111)
I20260811 18:59:12.391554 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 32510
I20260811 18:59:12.411325 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 31832
2026-08-11T18:59:12Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2460 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-11T18:59:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:59:12Z Disabled control of system clock
I20260811 18:59:12.448748 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474496157106-28370-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474496157106-28370-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474496157106-28370-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.190:46215
--webserver_interface=127.27.180.190
--webserver_port=0
--builtin_ntp_servers=127.27.180.148:37273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.180.190:46215
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 18:59:12.560241 367 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:59:12.560505 367 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:59:12.560555 367 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:59:12.564852 367 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:59:12.564913 367 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:59:12.564934 367 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:59:12.564956 367 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:59:12.564985 367 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:59:12.569825 367 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:37273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474496157106-28370-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474496157106-28370-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.180.190:46215
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.190:46215
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.180.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.367
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474496157106-28370-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:59:12.570859 367 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:59:12.572192 367 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:59:12.580374 381 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:12.580406 382 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:59:12.580555 367 server_base.cc:1034] running on GCE node
W20260811 18:59:12.580403 384 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:59:12.581234 367 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:59:12.582237 367 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:59:12.583396 367 hybrid_clock.cc:648] HybridClock initialized: now 1786474752583383 us; error 26 us; skew 500 ppm
I20260811 18:59:12.585141 367 webserver.cc:460] Webserver started at http://127.27.180.190:37717/ using document root <none> and password file <none>
I20260811 18:59:12.585669 367 fs_manager.cc:359] Metadata directory not provided
I20260811 18:59:12.585736 367 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:59:12.585940 367 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:59:12.587896 367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474496157106-28370-0/minicluster-data/master-0/data/instance:
uuid: "8daf8a14d6ee47e7a65b532204a620f4"
format_stamp: "Formatted at 2026-08-11 18:59:12 on dist-test-slave-2j7r"
server_key: "62e54b064fc51c67650e47e86cef5026"
server_key_iv: "f0114164c12b596dfe5fd60c2b18edbf"
server_key_version: "encryptionkey@0"
I20260811 18:59:12.588419 367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474496157106-28370-0/minicluster-data/master-0/wal/instance:
uuid: "8daf8a14d6ee47e7a65b532204a620f4"
format_stamp: "Formatted at 2026-08-11 18:59:12 on dist-test-slave-2j7r"
server_key: "62e54b064fc51c67650e47e86cef5026"
server_key_iv: "f0114164c12b596dfe5fd60c2b18edbf"
server_key_version: "encryptionkey@0"
I20260811 18:59:12.592587 367 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260811 18:59:12.594882 397 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:12.596006 367 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 18:59:12.596118 367 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474496157106-28370-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474496157106-28370-0/minicluster-data/master-0/wal
uuid: "8daf8a14d6ee47e7a65b532204a620f4"
format_stamp: "Formatted at 2026-08-11 18:59:12 on dist-test-slave-2j7r"
server_key: "62e54b064fc51c67650e47e86cef5026"
server_key_iv: "f0114164c12b596dfe5fd60c2b18edbf"
server_key_version: "encryptionkey@0"
I20260811 18:59:12.596206 367 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474496157106-28370-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474496157106-28370-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474496157106-28370-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:59:12.616463 367 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:59:12.617113 367 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:59:12.617280 367 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:59:12.623760 367 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.190:46215
I20260811 18:59:12.623776 494 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.190:46215 every 8 connection(s)
I20260811 18:59:12.624624 367 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
I20260811 18:59:12.627683 496 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:59:12.632997 496 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8daf8a14d6ee47e7a65b532204a620f4: Bootstrap starting.
I20260811 18:59:12.634266 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 367
I20260811 18:59:12.634388 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474496157106-28370-0/minicluster-data/master-0/wal/instance
I20260811 18:59:12.634642 28370 external_mini_cluster.cc:1428] Setting key 48cf612c65ef364d4f246dc246c57a0c
I20260811 18:59:12.635560 496 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8daf8a14d6ee47e7a65b532204a620f4: Neither blocks nor log segments found. Creating new log.
I20260811 18:59:12.636396 496 log.cc:824] T 00000000000000000000000000000000 P 8daf8a14d6ee47e7a65b532204a620f4: Log is configured to *not* fsync() on all Append() calls
I20260811 18:59:12.638366 496 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8daf8a14d6ee47e7a65b532204a620f4: No bootstrap required, opened a new log
I20260811 18:59:12.641765 496 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8daf8a14d6ee47e7a65b532204a620f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8daf8a14d6ee47e7a65b532204a620f4" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 46215 } }
I20260811 18:59:12.641963 496 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8daf8a14d6ee47e7a65b532204a620f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:59:12.641997 496 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8daf8a14d6ee47e7a65b532204a620f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8daf8a14d6ee47e7a65b532204a620f4, State: Initialized, Role: FOLLOWER
I20260811 18:59:12.642438 496 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8daf8a14d6ee47e7a65b532204a620f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8daf8a14d6ee47e7a65b532204a620f4" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 46215 } }
I20260811 18:59:12.642593 496 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8daf8a14d6ee47e7a65b532204a620f4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:59:12.642660 496 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8daf8a14d6ee47e7a65b532204a620f4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:59:12.642755 496 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8daf8a14d6ee47e7a65b532204a620f4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:59:12.643340 500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:12.635673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40622 (local address 127.27.180.190:46215)
0811 18:59:12.636325 (+ 652us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:12.636339 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:12.636388 (+ 49us) server_negotiation.cc:407] Connection header received
0811 18:59:12.637146 (+ 758us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:12.637171 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:12.637525 (+ 354us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:12.637813 (+ 288us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:12.638218 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:12.639165 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:12.640068 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:12.640564 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:12.640709 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:12.641577 (+ 868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:12.641582 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:12.641594 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:59:12.641969 (+ 375us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:12.642040 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:59:12.642068 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:12.642155 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:12.642298 (+ 143us) server_negotiation.cc:299] Negotiation successful
0811 18:59:12.642560 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":133,"threads_started":1}
I20260811 18:59:12.643605 496 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8daf8a14d6ee47e7a65b532204a620f4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8daf8a14d6ee47e7a65b532204a620f4" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 46215 } }
I20260811 18:59:12.643931 496 leader_election.cc:302] T 00000000000000000000000000000000 P 8daf8a14d6ee47e7a65b532204a620f4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8daf8a14d6ee47e7a65b532204a620f4; no voters:
I20260811 18:59:12.644198 496 leader_election.cc:288] T 00000000000000000000000000000000 P 8daf8a14d6ee47e7a65b532204a620f4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:59:12.644279 502 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8daf8a14d6ee47e7a65b532204a620f4 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:59:12.644460 502 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8daf8a14d6ee47e7a65b532204a620f4 [term 1 LEADER]: Becoming Leader. State: Replica: 8daf8a14d6ee47e7a65b532204a620f4, State: Running, Role: LEADER
I20260811 18:59:12.644742 502 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8daf8a14d6ee47e7a65b532204a620f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8daf8a14d6ee47e7a65b532204a620f4" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 46215 } }
I20260811 18:59:12.645453 496 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8daf8a14d6ee47e7a65b532204a620f4 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:59:12.646433 506 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8daf8a14d6ee47e7a65b532204a620f4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8daf8a14d6ee47e7a65b532204a620f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8daf8a14d6ee47e7a65b532204a620f4" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 46215 } } }
I20260811 18:59:12.646438 505 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8daf8a14d6ee47e7a65b532204a620f4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8daf8a14d6ee47e7a65b532204a620f4. Latest consensus state: current_term: 1 leader_uuid: "8daf8a14d6ee47e7a65b532204a620f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8daf8a14d6ee47e7a65b532204a620f4" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 46215 } } }
I20260811 18:59:12.646582 506 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8daf8a14d6ee47e7a65b532204a620f4 [sys.catalog]: This master's current role is: LEADER
I20260811 18:59:12.646582 505 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8daf8a14d6ee47e7a65b532204a620f4 [sys.catalog]: This master's current role is: LEADER
I20260811 18:59:12.646956 522 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:59:12.650045 522 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:59:12.655390 522 catalog_manager.cc:1347] Generated new cluster ID: 1a69cb0640a34201a7bd587465281378
I20260811 18:59:12.655452 522 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:59:12.671155 522 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:59:12.672333 522 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:59:12.678730 522 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8daf8a14d6ee47e7a65b532204a620f4: Generated new TSK 0
I20260811 18:59:12.679401 522 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:59:12.691032 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474496157106-28370-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474496157106-28370-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474496157106-28370-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474496157106-28370-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.180.129:0
--local_ip_for_outbound_sockets=127.27.180.129
--webserver_interface=127.27.180.129
--webserver_port=0
--tserver_master_addrs=127.27.180.190:46215
--builtin_ntp_servers=127.27.180.148:37273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 18:59:12.804042 543 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:59:12.804301 543 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:59:12.804342 543 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:59:12.808292 543 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:59:12.808343 543 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:59:12.808399 543 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.180.129
I20260811 18:59:12.812952 543 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:37273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474496157106-28370-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474496157106-28370-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.180.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474496157106-28370-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.27.180.129
--webserver_port=0
--tserver_master_addrs=127.27.180.190:46215
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.543
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.180.129
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474496157106-28370-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:59:12.813930 543 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:59:12.815224 543 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 18:59:12.824698 543 server_base.cc:1034] running on GCE node
W20260811 18:59:12.824743 549 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:12.824760 551 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:12.824743 553 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:59:12.825343 543 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:59:12.825960 543 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:59:12.827116 543 hybrid_clock.cc:648] HybridClock initialized: now 1786474752827101 us; error 25 us; skew 500 ppm
I20260811 18:59:12.832508 543 webserver.cc:460] Webserver started at http://127.27.180.129:44097/ using document root <none> and password file <none>
I20260811 18:59:12.833038 543 fs_manager.cc:359] Metadata directory not provided
I20260811 18:59:12.833130 543 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:59:12.833320 543 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:59:12.835273 543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474496157106-28370-0/minicluster-data/ts-0/data/instance:
uuid: "17561db659174f0a8786b5aa0b0f795d"
format_stamp: "Formatted at 2026-08-11 18:59:12 on dist-test-slave-2j7r"
server_key: "02c8cf576aeaea2e6e471020b3280bc4"
server_key_iv: "9a5638b132f85edcd8b9f560672d2979"
server_key_version: "encryptionkey@0"
I20260811 18:59:12.835783 543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474496157106-28370-0/minicluster-data/ts-0/wal/instance:
uuid: "17561db659174f0a8786b5aa0b0f795d"
format_stamp: "Formatted at 2026-08-11 18:59:12 on dist-test-slave-2j7r"
server_key: "02c8cf576aeaea2e6e471020b3280bc4"
server_key_iv: "9a5638b132f85edcd8b9f560672d2979"
server_key_version: "encryptionkey@0"
I20260811 18:59:12.839229 543 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260811 18:59:12.841557 566 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:12.842736 543 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 18:59:12.842833 543 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474496157106-28370-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474496157106-28370-0/minicluster-data/ts-0/wal
uuid: "17561db659174f0a8786b5aa0b0f795d"
format_stamp: "Formatted at 2026-08-11 18:59:12 on dist-test-slave-2j7r"
server_key: "02c8cf576aeaea2e6e471020b3280bc4"
server_key_iv: "9a5638b132f85edcd8b9f560672d2979"
server_key_version: "encryptionkey@0"
I20260811 18:59:12.842939 543 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474496157106-28370-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474496157106-28370-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474496157106-28370-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:59:12.857767 543 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:59:12.858368 543 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:59:12.858520 543 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:59:12.859040 543 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:59:12.859899 543 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:59:12.859944 543 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:12.859984 543 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:59:12.860013 543 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:12.870260 543 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.129:38481
I20260811 18:59:12.871117 543 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474496157106-28370-0/minicluster-data/ts-0/data/info.pb
I20260811 18:59:12.871735 784 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.129:38481 every 8 connection(s)
I20260811 18:59:12.878755 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 543
I20260811 18:59:12.878881 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474496157106-28370-0/minicluster-data/ts-0/wal/instance
I20260811 18:59:12.879200 28370 external_mini_cluster.cc:1428] Setting key 28e2e57d40c0c004446d3a0a990221ee
I20260811 18:59:12.879577 500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:12.872835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.129:36341 (local address 127.27.180.190:46215)
0811 18:59:12.872963 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:12.872967 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:12.873991 (+ 1024us) server_negotiation.cc:407] Connection header received
0811 18:59:12.875318 (+ 1327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:12.875321 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:12.875368 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:12.875438 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:12.876210 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:12.876730 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:12.877786 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:12.878009 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:12.878086 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:12.879120 (+ 1034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:12.879124 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:12.879127 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:59:12.879141 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:12.879166 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:59:12.879177 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:12.879258 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:12.879378 (+ 120us) server_negotiation.cc:299] Negotiation successful
0811 18:59:12.879440 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 18:59:12.880330 789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:12.873253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:46215 (local address 127.27.180.129:36341)
0811 18:59:12.873872 (+ 619us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:12.873909 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 18:59:12.875130 (+ 1221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:59:12.875574 (+ 444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:59:12.875580 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:59:12.875656 (+ 76us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:59:12.876110 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:12.876121 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:12.876814 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:12.876816 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:12.877688 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:12.877698 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:12.878117 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:12.878120 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:12.878278 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:12.878846 (+ 568us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:59:12.878848 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:59:12.878898 (+ 50us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:59:12.878971 (+ 73us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:59:12.879253 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:59:12.879261 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:59:12.879266 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 18:59:12.879362 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 18:59:12.879560 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":111,"threads_started":1}
I20260811 18:59:12.881228 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474496157106-28370-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474496157106-28370-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474496157106-28370-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474496157106-28370-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.180.130:0
--local_ip_for_outbound_sockets=127.27.180.130
--webserver_interface=127.27.180.130
--webserver_port=0
--tserver_master_addrs=127.27.180.190:46215
--builtin_ntp_servers=127.27.180.148:37273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 18:59:12.881465 785 heartbeater.cc:347] Connected to a master server at 127.27.180.190:46215
I20260811 18:59:12.881664 785 heartbeater.cc:464] Registering TS with master...
I20260811 18:59:12.882135 785 heartbeater.cc:511] Master 127.27.180.190:46215 requested a full tablet report, sending...
I20260811 18:59:12.883692 428 ts_manager.cc:194] Registered new tserver with Master: 17561db659174f0a8786b5aa0b0f795d (127.27.180.129:38481)
I20260811 18:59:12.885136 428 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.27.180.129:36341
W20260811 18:59:12.990886 790 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:59:12.991106 790 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:59:12.991142 790 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:59:12.995013 790 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:59:12.995055 790 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:59:12.995119 790 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.180.130
I20260811 18:59:12.999699 790 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:37273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474496157106-28370-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474496157106-28370-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.180.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474496157106-28370-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.27.180.130
--webserver_port=0
--tserver_master_addrs=127.27.180.190:46215
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.790
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.180.130
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474496157106-28370-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:59:13.000671 790 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:59:13.001971 790 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:59:13.011112 805 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:13.011226 801 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:13.011457 802 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:59:13.011432 790 server_base.cc:1034] running on GCE node
I20260811 18:59:13.011904 790 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:59:13.012549 790 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:59:13.013720 790 hybrid_clock.cc:648] HybridClock initialized: now 1786474753013703 us; error 28 us; skew 500 ppm
I20260811 18:59:13.015404 790 webserver.cc:460] Webserver started at http://127.27.180.130:45093/ using document root <none> and password file <none>
I20260811 18:59:13.015929 790 fs_manager.cc:359] Metadata directory not provided
I20260811 18:59:13.015993 790 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:59:13.016191 790 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:59:13.018205 790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474496157106-28370-0/minicluster-data/ts-1/data/instance:
uuid: "00fee563dc964ab98cce0e5cc74bdb56"
format_stamp: "Formatted at 2026-08-11 18:59:13 on dist-test-slave-2j7r"
server_key: "4f6098a066b807a913923a8fe6614217"
server_key_iv: "0f4b1a9e90dd0a9149cc26e57a2ac676"
server_key_version: "encryptionkey@0"
I20260811 18:59:13.018719 790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474496157106-28370-0/minicluster-data/ts-1/wal/instance:
uuid: "00fee563dc964ab98cce0e5cc74bdb56"
format_stamp: "Formatted at 2026-08-11 18:59:13 on dist-test-slave-2j7r"
server_key: "4f6098a066b807a913923a8fe6614217"
server_key_iv: "0f4b1a9e90dd0a9149cc26e57a2ac676"
server_key_version: "encryptionkey@0"
I20260811 18:59:13.022333 790 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 18:59:13.024565 817 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:13.025748 790 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 18:59:13.025876 790 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474496157106-28370-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474496157106-28370-0/minicluster-data/ts-1/wal
uuid: "00fee563dc964ab98cce0e5cc74bdb56"
format_stamp: "Formatted at 2026-08-11 18:59:13 on dist-test-slave-2j7r"
server_key: "4f6098a066b807a913923a8fe6614217"
server_key_iv: "0f4b1a9e90dd0a9149cc26e57a2ac676"
server_key_version: "encryptionkey@0"
I20260811 18:59:13.025985 790 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474496157106-28370-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474496157106-28370-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474496157106-28370-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:59:13.043208 790 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:59:13.043843 790 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:59:13.044031 790 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:59:13.044600 790 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:59:13.045655 790 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:59:13.045719 790 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:13.045799 790 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:59:13.045835 790 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:13.055308 790 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.130:35011
I20260811 18:59:13.055334 1029 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.130:35011 every 8 connection(s)
I20260811 18:59:13.056247 790 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474496157106-28370-0/minicluster-data/ts-1/data/info.pb
I20260811 18:59:13.058699 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 790
I20260811 18:59:13.058851 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474496157106-28370-0/minicluster-data/ts-1/wal/instance
I20260811 18:59:13.059175 28370 external_mini_cluster.cc:1428] Setting key 654ab28a4c922d8339b810a5cc4b683d
I20260811 18:59:13.061131 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474496157106-28370-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474496157106-28370-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474496157106-28370-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474496157106-28370-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.27.180.131:0
--local_ip_for_outbound_sockets=127.27.180.131
--webserver_interface=127.27.180.131
--webserver_port=0
--tserver_master_addrs=127.27.180.190:46215
--builtin_ntp_servers=127.27.180.148:37273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 18:59:13.065460 500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:13.058118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.130:44567 (local address 127.27.180.190:46215)
0811 18:59:13.058255 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:13.058259 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:13.059417 (+ 1158us) server_negotiation.cc:407] Connection header received
0811 18:59:13.060908 (+ 1491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:13.060911 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:13.060965 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:13.061038 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:13.061992 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:13.062512 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:13.063642 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:13.063866 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:13.063956 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:13.064973 (+ 1017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:13.064976 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:13.064979 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:59:13.064991 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:13.065011 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:59:13.065018 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:13.065077 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:13.065246 (+ 169us) server_negotiation.cc:299] Negotiation successful
0811 18:59:13.065313 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 18:59:13.066300 1037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:13.058504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:46215 (local address 127.27.180.130:44567)
0811 18:59:13.059253 (+ 749us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:13.059305 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 18:59:13.060710 (+ 1405us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:59:13.061250 (+ 540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:59:13.061261 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:59:13.061374 (+ 113us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:59:13.061851 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:13.061866 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:13.062638 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:13.062642 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:13.063522 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:13.063532 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:13.064014 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:13.064019 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:13.064172 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:13.064720 (+ 548us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:59:13.064721 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:59:13.064762 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:59:13.064826 (+ 64us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:59:13.065112 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:59:13.065122 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:59:13.065128 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 18:59:13.065236 (+ 108us) client_negotiation.cc:240] Negotiation successful
0811 18:59:13.065523 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":163,"threads_started":1}
I20260811 18:59:13.067467 1035 heartbeater.cc:347] Connected to a master server at 127.27.180.190:46215
I20260811 18:59:13.067694 1035 heartbeater.cc:464] Registering TS with master...
I20260811 18:59:13.068212 1035 heartbeater.cc:511] Master 127.27.180.190:46215 requested a full tablet report, sending...
I20260811 18:59:13.069325 428 ts_manager.cc:194] Registered new tserver with Master: 00fee563dc964ab98cce0e5cc74bdb56 (127.27.180.130:35011)
I20260811 18:59:13.070084 428 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.27.180.130:44567
W20260811 18:59:13.171011 1040 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:59:13.171234 1040 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:59:13.171281 1040 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:59:13.175145 1040 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:59:13.175189 1040 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:59:13.175237 1040 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.27.180.131
I20260811 18:59:13.179807 1040 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:37273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474496157106-28370-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474496157106-28370-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.27.180.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474496157106-28370-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.27.180.131
--webserver_port=0
--tserver_master_addrs=127.27.180.190:46215
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1040
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.27.180.131
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474496157106-28370-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:59:13.180819 1040 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:59:13.182080 1040 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:59:13.191090 1052 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:13.191084 1049 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:13.191095 1050 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:59:13.191308 1040 server_base.cc:1034] running on GCE node
I20260811 18:59:13.191731 1040 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:59:13.192301 1040 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:59:13.193445 1040 hybrid_clock.cc:648] HybridClock initialized: now 1786474753193423 us; error 36 us; skew 500 ppm
I20260811 18:59:13.195077 1040 webserver.cc:460] Webserver started at http://127.27.180.131:46509/ using document root <none> and password file <none>
I20260811 18:59:13.195596 1040 fs_manager.cc:359] Metadata directory not provided
I20260811 18:59:13.195652 1040 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:59:13.195809 1040 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:59:13.197782 1040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474496157106-28370-0/minicluster-data/ts-2/data/instance:
uuid: "e6b4c4d14a144b99b49606d973928750"
format_stamp: "Formatted at 2026-08-11 18:59:13 on dist-test-slave-2j7r"
server_key: "585077518966b595a880b673247024b8"
server_key_iv: "94152fa00adf730140959724f8385317"
server_key_version: "encryptionkey@0"
I20260811 18:59:13.198279 1040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474496157106-28370-0/minicluster-data/ts-2/wal/instance:
uuid: "e6b4c4d14a144b99b49606d973928750"
format_stamp: "Formatted at 2026-08-11 18:59:13 on dist-test-slave-2j7r"
server_key: "585077518966b595a880b673247024b8"
server_key_iv: "94152fa00adf730140959724f8385317"
server_key_version: "encryptionkey@0"
I20260811 18:59:13.201794 1040 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:59:13.203999 1062 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:13.205164 1040 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260811 18:59:13.205269 1040 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474496157106-28370-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474496157106-28370-0/minicluster-data/ts-2/wal
uuid: "e6b4c4d14a144b99b49606d973928750"
format_stamp: "Formatted at 2026-08-11 18:59:13 on dist-test-slave-2j7r"
server_key: "585077518966b595a880b673247024b8"
server_key_iv: "94152fa00adf730140959724f8385317"
server_key_version: "encryptionkey@0"
I20260811 18:59:13.205363 1040 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474496157106-28370-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474496157106-28370-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474496157106-28370-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:59:13.231235 1040 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:59:13.231952 1040 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:59:13.232128 1040 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:59:13.232744 1040 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 18:59:13.233747 1040 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 18:59:13.233793 1040 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:13.233835 1040 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 18:59:13.233865 1040 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:13.242175 1040 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.131:37213
I20260811 18:59:13.242218 1286 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.131:37213 every 8 connection(s)
I20260811 18:59:13.243026 1040 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474496157106-28370-0/minicluster-data/ts-2/data/info.pb
I20260811 18:59:13.247860 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 1040
I20260811 18:59:13.247958 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474496157106-28370-0/minicluster-data/ts-2/wal/instance
I20260811 18:59:13.248219 28370 external_mini_cluster.cc:1428] Setting key 727a5d7ba34c9fbf82aa9c590e5a0e92
I20260811 18:59:13.252257 500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:13.244754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.131:41511 (local address 127.27.180.190:46215)
0811 18:59:13.244889 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:13.244893 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:13.245979 (+ 1086us) server_negotiation.cc:407] Connection header received
0811 18:59:13.247611 (+ 1632us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:13.247615 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:13.247674 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:13.247750 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:13.248720 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:13.249333 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:13.250447 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:13.250667 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:13.250757 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:13.251792 (+ 1035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:13.251795 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:13.251797 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:59:13.251809 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:13.251827 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:59:13.251834 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:13.251900 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:13.252050 (+ 150us) server_negotiation.cc:299] Negotiation successful
0811 18:59:13.252111 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 18:59:13.253058 1291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:13.245150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:46215 (local address 127.27.180.131:41511)
0811 18:59:13.245819 (+ 669us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:13.245863 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 18:59:13.247407 (+ 1544us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:59:13.247942 (+ 535us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:59:13.247951 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:59:13.248059 (+ 108us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:59:13.248570 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:13.248584 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:13.249460 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:13.249464 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:13.250316 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:13.250325 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:13.250810 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:13.250814 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:13.250972 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:13.251542 (+ 570us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 18:59:13.251544 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:59:13.251582 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 18:59:13.251644 (+ 62us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:59:13.251913 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:59:13.251923 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:59:13.251929 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 18:59:13.252035 (+ 106us) client_negotiation.cc:240] Negotiation successful
0811 18:59:13.252312 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":102,"threads_started":1}
I20260811 18:59:13.254086 1287 heartbeater.cc:347] Connected to a master server at 127.27.180.190:46215
I20260811 18:59:13.254284 1287 heartbeater.cc:464] Registering TS with master...
I20260811 18:59:13.254750 1287 heartbeater.cc:511] Master 127.27.180.190:46215 requested a full tablet report, sending...
I20260811 18:59:13.255659 428 ts_manager.cc:194] Registered new tserver with Master: e6b4c4d14a144b99b49606d973928750 (127.27.180.131:37213)
I20260811 18:59:13.256186 428 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.27.180.131:41511
I20260811 18:59:13.262142 28370 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 18:59:13.267184 500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:13.263997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40626 (local address 127.27.180.190:46215)
0811 18:59:13.264102 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:13.264105 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:13.264256 (+ 151us) server_negotiation.cc:407] Connection header received
0811 18:59:13.264349 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:13.264352 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:13.264390 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:13.264452 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:13.264658 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:13.265177 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:13.265956 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:13.266148 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:13.266215 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:13.266882 (+ 667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:13.266885 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:13.266887 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:59:13.266899 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:13.266916 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:59:13.266923 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:13.266978 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:13.267038 (+ 60us) server_negotiation.cc:299] Negotiation successful
0811 18:59:13.267085 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 18:59:13.269596 428 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40626:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 18:59:13.272399 428 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 18:59:13.285516 1313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:13.280045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60030 (local address 127.27.180.131:37213)
0811 18:59:13.280760 (+ 715us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:13.280771 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:13.280796 (+ 25us) server_negotiation.cc:407] Connection header received
0811 18:59:13.281006 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:13.281011 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:13.281211 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:13.281393 (+ 182us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:59:13.281837 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:13.282744 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:13.284052 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:13.284828 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:13.284939 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:13.285150 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:13.285275 (+ 125us) server_negotiation.cc:299] Negotiation successful
0811 18:59:13.285378 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":614,"thread_start_us":147,"threads_started":1}
I20260811 18:59:13.285501 1310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:13.279930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.131:37213 (local address 127.0.0.1:60030)
0811 18:59:13.280655 (+ 725us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:13.280703 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 18:59:13.280816 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:59:13.281563 (+ 747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:59:13.281568 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:59:13.281592 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:59:13.281710 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:13.281716 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:13.282849 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:13.282852 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:13.283911 (+ 1059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:13.283922 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:13.284943 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:13.284947 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:13.285063 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:13.285128 (+ 65us) client_negotiation.cc:769] Sending connection context
0811 18:59:13.285233 (+ 105us) client_negotiation.cc:240] Negotiation successful
0811 18:59:13.285356 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":576,"thread_start_us":80,"threads_started":1}
I20260811 18:59:13.285909 1311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:13.280021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.129:38481 (local address 127.0.0.1:49988)
0811 18:59:13.280720 (+ 699us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:13.280733 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:59:13.280825 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:59:13.281932 (+ 1107us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:59:13.281934 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:59:13.281951 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:59:13.282046 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:13.282052 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:13.283475 (+ 1423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:13.283479 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:13.284565 (+ 1086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:13.284575 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:13.285543 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:13.285546 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:13.285631 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:13.285655 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 18:59:13.285705 (+ 50us) client_negotiation.cc:240] Negotiation successful
0811 18:59:13.285781 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":636,"thread_start_us":80,"threads_started":1}
I20260811 18:59:13.286093 1316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:13.280740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.130:35011 (local address 127.0.0.1:49366)
0811 18:59:13.281354 (+ 614us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:13.281366 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 18:59:13.281446 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:59:13.282108 (+ 662us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:59:13.282111 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:59:13.282130 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:59:13.282208 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:13.282214 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:13.283518 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:13.283520 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:13.284492 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:13.284502 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:13.285738 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:13.285741 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:13.285815 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:13.285835 (+ 20us) client_negotiation.cc:769] Sending connection context
0811 18:59:13.285907 (+ 72us) client_negotiation.cc:240] Negotiation successful
0811 18:59:13.285983 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":555,"thread_start_us":91,"threads_started":1}
I20260811 18:59:13.286666 1314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:13.280223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49988 (local address 127.27.180.129:38481)
0811 18:59:13.280997 (+ 774us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:13.281009 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:13.281030 (+ 21us) server_negotiation.cc:407] Connection header received
0811 18:59:13.281128 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:13.281133 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:13.281792 (+ 659us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:13.281943 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:59:13.282171 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:13.283343 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:13.284668 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:13.285424 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:13.286292 (+ 868us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:13.286410 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:13.286446 (+ 36us) server_negotiation.cc:299] Negotiation successful
0811 18:59:13.286511 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":667,"thread_start_us":88,"threads_started":1}
I20260811 18:59:13.286837 1318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:13.281208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49366 (local address 127.27.180.130:35011)
0811 18:59:13.281648 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:13.281659 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:13.281681 (+ 22us) server_negotiation.cc:407] Connection header received
0811 18:59:13.281765 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:13.281771 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:13.281938 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:13.282075 (+ 137us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:59:13.282327 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:13.283403 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:13.284603 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:13.285401 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:13.285524 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:13.285679 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:13.286606 (+ 927us) server_negotiation.cc:299] Negotiation successful
0811 18:59:13.286684 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":347,"thread_start_us":127,"threads_started":1}
I20260811 18:59:13.288795 1157 tablet_service.cc:1467] Processing CreateTablet for tablet 3f66a2ff27e44477bda3e4a315617109 (DEFAULT_TABLE table=test-table [id=27def0cf85a84b4886a785997a23758d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:59:13.289211 906 tablet_service.cc:1467] Processing CreateTablet for tablet 3f66a2ff27e44477bda3e4a315617109 (DEFAULT_TABLE table=test-table [id=27def0cf85a84b4886a785997a23758d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:59:13.289260 655 tablet_service.cc:1467] Processing CreateTablet for tablet 3f66a2ff27e44477bda3e4a315617109 (DEFAULT_TABLE table=test-table [id=27def0cf85a84b4886a785997a23758d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 18:59:13.289956 1157 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f66a2ff27e44477bda3e4a315617109. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:59:13.290092 906 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f66a2ff27e44477bda3e4a315617109. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:59:13.290174 655 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f66a2ff27e44477bda3e4a315617109. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:59:13.296489 1319 tablet_bootstrap.cc:492] T 3f66a2ff27e44477bda3e4a315617109 P 00fee563dc964ab98cce0e5cc74bdb56: Bootstrap starting.
I20260811 18:59:13.297169 1322 tablet_bootstrap.cc:492] T 3f66a2ff27e44477bda3e4a315617109 P 17561db659174f0a8786b5aa0b0f795d: Bootstrap starting.
I20260811 18:59:13.297588 1320 tablet_bootstrap.cc:492] T 3f66a2ff27e44477bda3e4a315617109 P e6b4c4d14a144b99b49606d973928750: Bootstrap starting.
I20260811 18:59:13.298840 1319 tablet_bootstrap.cc:654] T 3f66a2ff27e44477bda3e4a315617109 P 00fee563dc964ab98cce0e5cc74bdb56: Neither blocks nor log segments found. Creating new log.
I20260811 18:59:13.299345 1322 tablet_bootstrap.cc:654] T 3f66a2ff27e44477bda3e4a315617109 P 17561db659174f0a8786b5aa0b0f795d: Neither blocks nor log segments found. Creating new log.
I20260811 18:59:13.299515 1320 tablet_bootstrap.cc:654] T 3f66a2ff27e44477bda3e4a315617109 P e6b4c4d14a144b99b49606d973928750: Neither blocks nor log segments found. Creating new log.
I20260811 18:59:13.299788 1319 log.cc:824] T 3f66a2ff27e44477bda3e4a315617109 P 00fee563dc964ab98cce0e5cc74bdb56: Log is configured to *not* fsync() on all Append() calls
I20260811 18:59:13.300158 1322 log.cc:824] T 3f66a2ff27e44477bda3e4a315617109 P 17561db659174f0a8786b5aa0b0f795d: Log is configured to *not* fsync() on all Append() calls
I20260811 18:59:13.300239 1320 log.cc:824] T 3f66a2ff27e44477bda3e4a315617109 P e6b4c4d14a144b99b49606d973928750: Log is configured to *not* fsync() on all Append() calls
I20260811 18:59:13.302006 1319 tablet_bootstrap.cc:492] T 3f66a2ff27e44477bda3e4a315617109 P 00fee563dc964ab98cce0e5cc74bdb56: No bootstrap required, opened a new log
I20260811 18:59:13.302217 1319 ts_tablet_manager.cc:1397] T 3f66a2ff27e44477bda3e4a315617109 P 00fee563dc964ab98cce0e5cc74bdb56: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 18:59:13.302531 1322 tablet_bootstrap.cc:492] T 3f66a2ff27e44477bda3e4a315617109 P 17561db659174f0a8786b5aa0b0f795d: No bootstrap required, opened a new log
I20260811 18:59:13.302719 1322 ts_tablet_manager.cc:1397] T 3f66a2ff27e44477bda3e4a315617109 P 17561db659174f0a8786b5aa0b0f795d: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 18:59:13.302955 1320 tablet_bootstrap.cc:492] T 3f66a2ff27e44477bda3e4a315617109 P e6b4c4d14a144b99b49606d973928750: No bootstrap required, opened a new log
I20260811 18:59:13.303139 1320 ts_tablet_manager.cc:1397] T 3f66a2ff27e44477bda3e4a315617109 P e6b4c4d14a144b99b49606d973928750: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.001s
I20260811 18:59:13.306440 1319 raft_consensus.cc:348] T 3f66a2ff27e44477bda3e4a315617109 P 00fee563dc964ab98cce0e5cc74bdb56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b4c4d14a144b99b49606d973928750" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 37213 } } peers { permanent_uuid: "17561db659174f0a8786b5aa0b0f795d" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38481 } } peers { permanent_uuid: "00fee563dc964ab98cce0e5cc74bdb56" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 35011 } }
I20260811 18:59:13.306440 1322 raft_consensus.cc:348] T 3f66a2ff27e44477bda3e4a315617109 P 17561db659174f0a8786b5aa0b0f795d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b4c4d14a144b99b49606d973928750" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 37213 } } peers { permanent_uuid: "17561db659174f0a8786b5aa0b0f795d" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38481 } } peers { permanent_uuid: "00fee563dc964ab98cce0e5cc74bdb56" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 35011 } }
I20260811 18:59:13.306689 1319 raft_consensus.cc:374] T 3f66a2ff27e44477bda3e4a315617109 P 00fee563dc964ab98cce0e5cc74bdb56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:59:13.306689 1322 raft_consensus.cc:374] T 3f66a2ff27e44477bda3e4a315617109 P 17561db659174f0a8786b5aa0b0f795d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:59:13.306738 1322 raft_consensus.cc:729] T 3f66a2ff27e44477bda3e4a315617109 P 17561db659174f0a8786b5aa0b0f795d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17561db659174f0a8786b5aa0b0f795d, State: Initialized, Role: FOLLOWER
I20260811 18:59:13.306738 1319 raft_consensus.cc:729] T 3f66a2ff27e44477bda3e4a315617109 P 00fee563dc964ab98cce0e5cc74bdb56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00fee563dc964ab98cce0e5cc74bdb56, State: Initialized, Role: FOLLOWER
I20260811 18:59:13.306730 1320 raft_consensus.cc:348] T 3f66a2ff27e44477bda3e4a315617109 P e6b4c4d14a144b99b49606d973928750 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b4c4d14a144b99b49606d973928750" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 37213 } } peers { permanent_uuid: "17561db659174f0a8786b5aa0b0f795d" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38481 } } peers { permanent_uuid: "00fee563dc964ab98cce0e5cc74bdb56" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 35011 } }
I20260811 18:59:13.306924 1320 raft_consensus.cc:374] T 3f66a2ff27e44477bda3e4a315617109 P e6b4c4d14a144b99b49606d973928750 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:59:13.306963 1320 raft_consensus.cc:729] T 3f66a2ff27e44477bda3e4a315617109 P e6b4c4d14a144b99b49606d973928750 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6b4c4d14a144b99b49606d973928750, State: Initialized, Role: FOLLOWER
I20260811 18:59:13.307183 1319 consensus_queue.cc:260] T 3f66a2ff27e44477bda3e4a315617109 P 00fee563dc964ab98cce0e5cc74bdb56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b4c4d14a144b99b49606d973928750" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 37213 } } peers { permanent_uuid: "17561db659174f0a8786b5aa0b0f795d" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38481 } } peers { permanent_uuid: "00fee563dc964ab98cce0e5cc74bdb56" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 35011 } }
I20260811 18:59:13.307324 1322 consensus_queue.cc:260] T 3f66a2ff27e44477bda3e4a315617109 P 17561db659174f0a8786b5aa0b0f795d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b4c4d14a144b99b49606d973928750" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 37213 } } peers { permanent_uuid: "17561db659174f0a8786b5aa0b0f795d" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38481 } } peers { permanent_uuid: "00fee563dc964ab98cce0e5cc74bdb56" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 35011 } }
I20260811 18:59:13.307406 1320 consensus_queue.cc:260] T 3f66a2ff27e44477bda3e4a315617109 P e6b4c4d14a144b99b49606d973928750 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b4c4d14a144b99b49606d973928750" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 37213 } } peers { permanent_uuid: "17561db659174f0a8786b5aa0b0f795d" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38481 } } peers { permanent_uuid: "00fee563dc964ab98cce0e5cc74bdb56" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 35011 } }
I20260811 18:59:13.307958 785 heartbeater.cc:503] Master 127.27.180.190:46215 was elected leader, sending a full tablet report...
I20260811 18:59:13.308195 1319 ts_tablet_manager.cc:1428] T 3f66a2ff27e44477bda3e4a315617109 P 00fee563dc964ab98cce0e5cc74bdb56: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 18:59:13.308247 1322 ts_tablet_manager.cc:1428] T 3f66a2ff27e44477bda3e4a315617109 P 17561db659174f0a8786b5aa0b0f795d: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260811 18:59:13.308375 1320 ts_tablet_manager.cc:1428] T 3f66a2ff27e44477bda3e4a315617109 P e6b4c4d14a144b99b49606d973928750: Time spent starting tablet: real 0.005s user 0.002s sys 0.004s
I20260811 18:59:13.308495 1035 heartbeater.cc:503] Master 127.27.180.190:46215 was elected leader, sending a full tablet report...
I20260811 18:59:13.308694 1287 heartbeater.cc:503] Master 127.27.180.190:46215 was elected leader, sending a full tablet report...
W20260811 18:59:13.372212 786 tablet.cc:2369] T 3f66a2ff27e44477bda3e4a315617109 P 17561db659174f0a8786b5aa0b0f795d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 18:59:13.493989 1288 tablet.cc:2369] T 3f66a2ff27e44477bda3e4a315617109 P e6b4c4d14a144b99b49606d973928750: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:59:13.510998 1331 raft_consensus.cc:482] T 3f66a2ff27e44477bda3e4a315617109 P 17561db659174f0a8786b5aa0b0f795d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 18:59:13.511240 1331 raft_consensus.cc:504] T 3f66a2ff27e44477bda3e4a315617109 P 17561db659174f0a8786b5aa0b0f795d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b4c4d14a144b99b49606d973928750" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 37213 } } peers { permanent_uuid: "17561db659174f0a8786b5aa0b0f795d" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38481 } } peers { permanent_uuid: "00fee563dc964ab98cce0e5cc74bdb56" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 35011 } }
I20260811 18:59:13.512444 1331 leader_election.cc:288] T 3f66a2ff27e44477bda3e4a315617109 P 17561db659174f0a8786b5aa0b0f795d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e6b4c4d14a144b99b49606d973928750 (127.27.180.131:37213), 00fee563dc964ab98cce0e5cc74bdb56 (127.27.180.130:35011)
I20260811 18:59:13.516376 1313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:13.512858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.129:35321 (local address 127.27.180.131:37213)
0811 18:59:13.513003 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:13.513008 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:13.513030 (+ 22us) server_negotiation.cc:407] Connection header received
0811 18:59:13.513247 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:13.513263 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:13.513324 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:13.513372 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:59:13.513791 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:13.514339 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:13.515421 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:13.515822 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:13.515933 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:13.515985 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:13.516169 (+ 184us) server_negotiation.cc:299] Negotiation successful
0811 18:59:13.516246 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 18:59:13.516381 1318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:13.513123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.27.180.129:35869 (local address 127.27.180.130:35011)
0811 18:59:13.513252 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:13.513257 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:13.513317 (+ 60us) server_negotiation.cc:407] Connection header received
0811 18:59:13.513478 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:13.513481 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:13.513531 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:13.513604 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 18:59:13.513952 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:13.514473 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:13.515469 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:13.515850 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:13.515953 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:13.516001 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:13.516205 (+ 204us) server_negotiation.cc:299] Negotiation successful
0811 18:59:13.516276 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 18:59:13.516412 1334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:13.512661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.131:37213 (local address 127.27.180.129:35321)
0811 18:59:13.512962 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:13.512978 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 18:59:13.513107 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:59:13.513514 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:59:13.513516 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:59:13.513536 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:59:13.513669 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:13.513677 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:13.514448 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:13.514452 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:13.515295 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:13.515305 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:13.515946 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:13.515950 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:13.516062 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:13.516081 (+ 19us) client_negotiation.cc:769] Sending connection context
0811 18:59:13.516137 (+ 56us) client_negotiation.cc:240] Negotiation successful
0811 18:59:13.516201 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221,"mutex_wait_us":1,"thread_start_us":114,"threads_started":1}
I20260811 18:59:13.516415 1335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:13.512731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.130:35011 (local address 127.27.180.129:35869)
0811 18:59:13.513232 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:13.513245 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:59:13.513371 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:59:13.513628 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:59:13.513631 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:59:13.513651 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 18:59:13.513785 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:13.513793 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:13.514588 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:13.514590 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:13.515338 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:13.515357 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:13.515957 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:13.515961 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:13.516089 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:13.516107 (+ 18us) client_negotiation.cc:769] Sending connection context
0811 18:59:13.516182 (+ 75us) client_negotiation.cc:240] Negotiation successful
0811 18:59:13.516245 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":435,"mutex_wait_us":32,"thread_start_us":94,"threads_started":1}
I20260811 18:59:13.517261 942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f66a2ff27e44477bda3e4a315617109" candidate_uuid: "17561db659174f0a8786b5aa0b0f795d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00fee563dc964ab98cce0e5cc74bdb56" is_pre_election: true
I20260811 18:59:13.517261 1196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f66a2ff27e44477bda3e4a315617109" candidate_uuid: "17561db659174f0a8786b5aa0b0f795d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6b4c4d14a144b99b49606d973928750" is_pre_election: true
I20260811 18:59:13.517580 942 raft_consensus.cc:2454] T 3f66a2ff27e44477bda3e4a315617109 P 00fee563dc964ab98cce0e5cc74bdb56 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17561db659174f0a8786b5aa0b0f795d in term 0.
I20260811 18:59:13.517580 1196 raft_consensus.cc:2454] T 3f66a2ff27e44477bda3e4a315617109 P e6b4c4d14a144b99b49606d973928750 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17561db659174f0a8786b5aa0b0f795d in term 0.
I20260811 18:59:13.518129 571 leader_election.cc:302] T 3f66a2ff27e44477bda3e4a315617109 P 17561db659174f0a8786b5aa0b0f795d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00fee563dc964ab98cce0e5cc74bdb56, 17561db659174f0a8786b5aa0b0f795d; no voters:
I20260811 18:59:13.518390 1331 raft_consensus.cc:2781] T 3f66a2ff27e44477bda3e4a315617109 P 17561db659174f0a8786b5aa0b0f795d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 18:59:13.518471 1331 raft_consensus.cc:482] T 3f66a2ff27e44477bda3e4a315617109 P 17561db659174f0a8786b5aa0b0f795d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 18:59:13.518518 1331 raft_consensus.cc:3037] T 3f66a2ff27e44477bda3e4a315617109 P 17561db659174f0a8786b5aa0b0f795d [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:59:13.519547 1331 raft_consensus.cc:504] T 3f66a2ff27e44477bda3e4a315617109 P 17561db659174f0a8786b5aa0b0f795d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b4c4d14a144b99b49606d973928750" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 37213 } } peers { permanent_uuid: "17561db659174f0a8786b5aa0b0f795d" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38481 } } peers { permanent_uuid: "00fee563dc964ab98cce0e5cc74bdb56" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 35011 } }
I20260811 18:59:13.519874 1331 leader_election.cc:288] T 3f66a2ff27e44477bda3e4a315617109 P 17561db659174f0a8786b5aa0b0f795d [CANDIDATE]: Term 1 election: Requested vote from peers e6b4c4d14a144b99b49606d973928750 (127.27.180.131:37213), 00fee563dc964ab98cce0e5cc74bdb56 (127.27.180.130:35011)
I20260811 18:59:13.520184 942 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f66a2ff27e44477bda3e4a315617109" candidate_uuid: "17561db659174f0a8786b5aa0b0f795d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00fee563dc964ab98cce0e5cc74bdb56"
I20260811 18:59:13.520197 1196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f66a2ff27e44477bda3e4a315617109" candidate_uuid: "17561db659174f0a8786b5aa0b0f795d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6b4c4d14a144b99b49606d973928750"
I20260811 18:59:13.520298 942 raft_consensus.cc:3037] T 3f66a2ff27e44477bda3e4a315617109 P 00fee563dc964ab98cce0e5cc74bdb56 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:59:13.520325 1196 raft_consensus.cc:3037] T 3f66a2ff27e44477bda3e4a315617109 P e6b4c4d14a144b99b49606d973928750 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:59:13.521319 942 raft_consensus.cc:2454] T 3f66a2ff27e44477bda3e4a315617109 P 00fee563dc964ab98cce0e5cc74bdb56 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17561db659174f0a8786b5aa0b0f795d in term 1.
I20260811 18:59:13.521426 1196 raft_consensus.cc:2454] T 3f66a2ff27e44477bda3e4a315617109 P e6b4c4d14a144b99b49606d973928750 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17561db659174f0a8786b5aa0b0f795d in term 1.
I20260811 18:59:13.521647 571 leader_election.cc:302] T 3f66a2ff27e44477bda3e4a315617109 P 17561db659174f0a8786b5aa0b0f795d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00fee563dc964ab98cce0e5cc74bdb56, 17561db659174f0a8786b5aa0b0f795d; no voters:
I20260811 18:59:13.521852 1331 raft_consensus.cc:2781] T 3f66a2ff27e44477bda3e4a315617109 P 17561db659174f0a8786b5aa0b0f795d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:59:13.522046 1331 raft_consensus.cc:686] T 3f66a2ff27e44477bda3e4a315617109 P 17561db659174f0a8786b5aa0b0f795d [term 1 LEADER]: Becoming Leader. State: Replica: 17561db659174f0a8786b5aa0b0f795d, State: Running, Role: LEADER
I20260811 18:59:13.522393 1331 consensus_queue.cc:237] T 3f66a2ff27e44477bda3e4a315617109 P 17561db659174f0a8786b5aa0b0f795d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b4c4d14a144b99b49606d973928750" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 37213 } } peers { permanent_uuid: "17561db659174f0a8786b5aa0b0f795d" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38481 } } peers { permanent_uuid: "00fee563dc964ab98cce0e5cc74bdb56" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 35011 } }
I20260811 18:59:13.525419 421 catalog_manager.cc:5495] T 3f66a2ff27e44477bda3e4a315617109 P 17561db659174f0a8786b5aa0b0f795d reported cstate change: term changed from 0 to 1, leader changed from <none> to 17561db659174f0a8786b5aa0b0f795d (127.27.180.129). New cstate: current_term: 1 leader_uuid: "17561db659174f0a8786b5aa0b0f795d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6b4c4d14a144b99b49606d973928750" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 37213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17561db659174f0a8786b5aa0b0f795d" member_type: VOTER last_known_addr { host: "127.27.180.129" port: 38481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00fee563dc964ab98cce0e5cc74bdb56" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 35011 } health_report { overall_health: UNKNOWN } } }
W20260811 18:59:13.557413 1036 tablet.cc:2369] T 3f66a2ff27e44477bda3e4a315617109 P 00fee563dc964ab98cce0e5cc74bdb56: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:59:13.578614 1314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:13.575384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49990 (local address 127.27.180.129:38481)
0811 18:59:13.575525 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:13.575529 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:13.575545 (+ 16us) server_negotiation.cc:407] Connection header received
0811 18:59:13.575631 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:13.575634 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:13.575710 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:13.575807 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:59:13.575987 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:13.576495 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:13.577341 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:13.577567 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:13.577655 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:13.577872 (+ 217us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:59:13.578269 (+ 397us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:59:13.578354 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:13.578418 (+ 64us) server_negotiation.cc:299] Negotiation successful
0811 18:59:13.578469 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 18:59:13.584259 942 raft_consensus.cc:1261] T 3f66a2ff27e44477bda3e4a315617109 P 00fee563dc964ab98cce0e5cc74bdb56 [term 1 FOLLOWER]: Refusing update from remote peer 17561db659174f0a8786b5aa0b0f795d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:59:13.584256 1196 raft_consensus.cc:1261] T 3f66a2ff27e44477bda3e4a315617109 P e6b4c4d14a144b99b49606d973928750 [term 1 FOLLOWER]: Refusing update from remote peer 17561db659174f0a8786b5aa0b0f795d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:59:13.584925 1339 consensus_queue.cc:1035] T 3f66a2ff27e44477bda3e4a315617109 P 17561db659174f0a8786b5aa0b0f795d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6b4c4d14a144b99b49606d973928750" member_type: VOTER last_known_addr { host: "127.27.180.131" port: 37213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:59:13.585125 1331 consensus_queue.cc:1035] T 3f66a2ff27e44477bda3e4a315617109 P 17561db659174f0a8786b5aa0b0f795d [LEADER]: Connected to new peer: Peer: permanent_uuid: "00fee563dc964ab98cce0e5cc74bdb56" member_type: VOTER last_known_addr { host: "127.27.180.130" port: 35011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:59:13.590982 1348 mvcc.cc:205] Tried to move back new op lower bound from 7317400590674259968 to 7317400590430121984. Current Snapshot: MvccSnapshot[applied={T|T < 7317400590674259968}]
I20260811 18:59:13.599820 421 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:40626:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:59:13.599984 421 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:40626:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 18:59:13.602314 421 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8daf8a14d6ee47e7a65b532204a620f4: Sending DeleteTablet for 3 replicas of tablet 3f66a2ff27e44477bda3e4a315617109
I20260811 18:59:13.603027 1157 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f66a2ff27e44477bda3e4a315617109 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:59:13 UTC) from {username='slave'} at 127.0.0.1:60030
I20260811 18:59:13.603044 655 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f66a2ff27e44477bda3e4a315617109 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:59:13 UTC) from {username='slave'} at 127.0.0.1:49988
I20260811 18:59:13.603107 906 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f66a2ff27e44477bda3e4a315617109 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 18:59:13 UTC) from {username='slave'} at 127.0.0.1:49366
I20260811 18:59:13.603502 1364 tablet_replica.cc:330] T 3f66a2ff27e44477bda3e4a315617109 P e6b4c4d14a144b99b49606d973928750: stopping tablet replica
I20260811 18:59:13.603794 1364 raft_consensus.cc:2229] T 3f66a2ff27e44477bda3e4a315617109 P e6b4c4d14a144b99b49606d973928750 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:59:13.604056 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 543
I20260811 18:59:13.604079 1364 raft_consensus.cc:2258] T 3f66a2ff27e44477bda3e4a315617109 P e6b4c4d14a144b99b49606d973928750 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:59:13.637426 1362 tablet_replica.cc:330] T 3f66a2ff27e44477bda3e4a315617109 P 00fee563dc964ab98cce0e5cc74bdb56: stopping tablet replica
I20260811 18:59:13.637787 1362 raft_consensus.cc:2229] T 3f66a2ff27e44477bda3e4a315617109 P 00fee563dc964ab98cce0e5cc74bdb56 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 18:59:13.638123 1362 raft_consensus.cc:2258] T 3f66a2ff27e44477bda3e4a315617109 P 00fee563dc964ab98cce0e5cc74bdb56 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 18:59:13.639714 1364 ts_tablet_manager.cc:1905] T 3f66a2ff27e44477bda3e4a315617109 P e6b4c4d14a144b99b49606d973928750: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:59:13.641289 1362 ts_tablet_manager.cc:1905] T 3f66a2ff27e44477bda3e4a315617109 P 00fee563dc964ab98cce0e5cc74bdb56: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 18:59:13.642632 1364 ts_tablet_manager.cc:1918] T 3f66a2ff27e44477bda3e4a315617109 P e6b4c4d14a144b99b49606d973928750: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:59:13.642738 1364 log.cc:1196] T 3f66a2ff27e44477bda3e4a315617109 P e6b4c4d14a144b99b49606d973928750: Deleting WAL directory at /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474496157106-28370-0/minicluster-data/ts-2/wal/wals/3f66a2ff27e44477bda3e4a315617109
I20260811 18:59:13.643036 1364 ts_tablet_manager.cc:1939] T 3f66a2ff27e44477bda3e4a315617109 P e6b4c4d14a144b99b49606d973928750: Deleting consensus metadata
I20260811 18:59:13.644058 1362 ts_tablet_manager.cc:1918] T 3f66a2ff27e44477bda3e4a315617109 P 00fee563dc964ab98cce0e5cc74bdb56: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 18:59:13.644212 1362 log.cc:1196] T 3f66a2ff27e44477bda3e4a315617109 P 00fee563dc964ab98cce0e5cc74bdb56: Deleting WAL directory at /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786474496157106-28370-0/minicluster-data/ts-1/wal/wals/3f66a2ff27e44477bda3e4a315617109
I20260811 18:59:13.644384 402 catalog_manager.cc:4841] TS e6b4c4d14a144b99b49606d973928750 (127.27.180.131:37213): tablet 3f66a2ff27e44477bda3e4a315617109 (table test-table [id=27def0cf85a84b4886a785997a23758d]) successfully deleted
I20260811 18:59:13.644599 1362 ts_tablet_manager.cc:1939] T 3f66a2ff27e44477bda3e4a315617109 P 00fee563dc964ab98cce0e5cc74bdb56: Deleting consensus metadata
W20260811 18:59:13.645020 405 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.27.180.129:38481 (error 108)
I20260811 18:59:13.645787 407 catalog_manager.cc:4841] TS 00fee563dc964ab98cce0e5cc74bdb56 (127.27.180.130:35011): tablet 3f66a2ff27e44477bda3e4a315617109 (table test-table [id=27def0cf85a84b4886a785997a23758d]) successfully deleted
I20260811 18:59:13.646062 1316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:13.645726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.129:38481 (local address 127.0.0.1:50004)
0811 18:59:13.645871 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:13.645958 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.129:38481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260811 18:59:13.646158 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 790
W20260811 18:59:13.646221 405 catalog_manager.cc:4584] TS 17561db659174f0a8786b5aa0b0f795d (127.27.180.129:38481): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3f66a2ff27e44477bda3e4a315617109: Network error: Client connection negotiation failed: client connection to 127.27.180.129:38481: connect: Connection refused (error 111)
I20260811 18:59:13.663144 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 1040
I20260811 18:59:13.675045 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 367
2026-08-11T18:59:13Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1262 ms)
[----------] 6 tests from EncryptionPolicyTest (11199 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474496157106-28370-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:59:13 dist-test-slave-2j7r krb5kdc[1383](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:59:13 dist-test-slave-2j7r krb5kdc[1383](info): set up 2 sockets
Aug 11 18:59:13 dist-test-slave-2j7r krb5kdc[1383](info): commencing operation
krb5kdc: starting...
W20260811 18:59:14.882902 28370 mini_kdc.cc:121] Time spent starting KDC: real 1.180s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474496157106-28370-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474496157106-28370-0/krb5kdc/test-user.keytab.
Aug 11 18:59:14 dist-test-slave-2j7r krb5kdc[1383](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474754, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:59:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:59:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474496157106-28370-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474496157106-28370-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 18:59:15.004285 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474496157106-28370-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474496157106-28370-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474496157106-28370-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46571
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474496157106-28370-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46571
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:59:15.114044 1410 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:59:15.114281 1410 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:59:15.114319 1410 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:59:15.118182 1410 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:59:15.118232 1410 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:59:15.118250 1410 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:59:15.118264 1410 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:59:15.118278 1410 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:59:15.123420 1410 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474496157106-28370-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474496157106-28370-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46571
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474496157106-28370-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46571
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1410
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474496157106-28370-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:59:15.124434 1410 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:59:15.125452 1410 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:59:15.133421 1422 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:15.133492 1418 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:15.133491 1420 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:59:15.133548 1410 server_base.cc:1034] running on GCE node
I20260811 18:59:15.134284 1410 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:59:15.135311 1410 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:59:15.136474 1410 hybrid_clock.cc:648] HybridClock initialized: now 1786474755136453 us; error 32 us; skew 500 ppm
Aug 11 18:59:15 dist-test-slave-2j7r krb5kdc[1383](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474755, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:59:15.139122 1410 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 18:59:15.139953 1410 webserver.cc:460] Webserver started at http://127.0.0.1:42621/ using document root <none> and password file <none>
I20260811 18:59:15.140476 1410 fs_manager.cc:359] Metadata directory not provided
I20260811 18:59:15.140539 1410 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:59:15.140724 1410 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:59:15.142720 1410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474496157106-28370-0/minicluster-data/master-0/data/instance:
uuid: "d1c609779669456d8e14b87f86457ee3"
format_stamp: "Formatted at 2026-08-11 18:59:15 on dist-test-slave-2j7r"
server_key: "eacf83c3f7bf3e0e637006f44f82612d"
server_key_iv: "99ffe98fd156c32ff88973304ddf468d"
server_key_version: "encryptionkey@0"
I20260811 18:59:15.143254 1410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474496157106-28370-0/minicluster-data/master-0/wal/instance:
uuid: "d1c609779669456d8e14b87f86457ee3"
format_stamp: "Formatted at 2026-08-11 18:59:15 on dist-test-slave-2j7r"
server_key: "eacf83c3f7bf3e0e637006f44f82612d"
server_key_iv: "99ffe98fd156c32ff88973304ddf468d"
server_key_version: "encryptionkey@0"
I20260811 18:59:15.146777 1410 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260811 18:59:15.148962 1440 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:15.150120 1410 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 18:59:15.150240 1410 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474496157106-28370-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474496157106-28370-0/minicluster-data/master-0/wal
uuid: "d1c609779669456d8e14b87f86457ee3"
format_stamp: "Formatted at 2026-08-11 18:59:15 on dist-test-slave-2j7r"
server_key: "eacf83c3f7bf3e0e637006f44f82612d"
server_key_iv: "99ffe98fd156c32ff88973304ddf468d"
server_key_version: "encryptionkey@0"
I20260811 18:59:15.150329 1410 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474496157106-28370-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474496157106-28370-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474496157106-28370-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:59:15.180073 1410 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:59:15.182790 1410 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:59:15.182967 1410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:59:15.189808 1410 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46571
I20260811 18:59:15.189827 1528 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46571 every 8 connection(s)
I20260811 18:59:15.190716 1410 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
I20260811 18:59:15.193395 1530 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:59:15.198724 1530 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1c609779669456d8e14b87f86457ee3: Bootstrap starting.
I20260811 18:59:15.200843 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 1410
I20260811 18:59:15.200999 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786474496157106-28370-0/minicluster-data/master-0/wal/instance
I20260811 18:59:15.201331 28370 external_mini_cluster.cc:1428] Setting key c0e5a9e9dd951424495a2cde65a84b07
I20260811 18:59:15.201463 1530 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d1c609779669456d8e14b87f86457ee3: Neither blocks nor log segments found. Creating new log.
I20260811 18:59:15.202301 1530 log.cc:824] T 00000000000000000000000000000000 P d1c609779669456d8e14b87f86457ee3: Log is configured to *not* fsync() on all Append() calls
I20260811 18:59:15.204677 1530 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1c609779669456d8e14b87f86457ee3: No bootstrap required, opened a new log
I20260811 18:59:15.209000 1530 raft_consensus.cc:348] T 00000000000000000000000000000000 P d1c609779669456d8e14b87f86457ee3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1c609779669456d8e14b87f86457ee3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46571 } }
I20260811 18:59:15.209280 1530 raft_consensus.cc:374] T 00000000000000000000000000000000 P d1c609779669456d8e14b87f86457ee3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:59:15.209368 1530 raft_consensus.cc:729] T 00000000000000000000000000000000 P d1c609779669456d8e14b87f86457ee3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1c609779669456d8e14b87f86457ee3, State: Initialized, Role: FOLLOWER
I20260811 18:59:15.209806 1530 consensus_queue.cc:260] T 00000000000000000000000000000000 P d1c609779669456d8e14b87f86457ee3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1c609779669456d8e14b87f86457ee3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46571 } }
I20260811 18:59:15.209951 1530 raft_consensus.cc:388] T 00000000000000000000000000000000 P d1c609779669456d8e14b87f86457ee3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:59:15.210019 1530 raft_consensus.cc:482] T 00000000000000000000000000000000 P d1c609779669456d8e14b87f86457ee3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:59:15.210116 1530 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d1c609779669456d8e14b87f86457ee3 [term 0 FOLLOWER]: Advancing to term 1
Aug 11 18:59:15 dist-test-slave-2j7r krb5kdc[1383](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474754, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 18:59:15.211290 1530 raft_consensus.cc:504] T 00000000000000000000000000000000 P d1c609779669456d8e14b87f86457ee3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1c609779669456d8e14b87f86457ee3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46571 } }
I20260811 18:59:15.211637 1530 leader_election.cc:302] T 00000000000000000000000000000000 P d1c609779669456d8e14b87f86457ee3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d1c609779669456d8e14b87f86457ee3; no voters:
I20260811 18:59:15.211946 1530 leader_election.cc:288] T 00000000000000000000000000000000 P d1c609779669456d8e14b87f86457ee3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:59:15.212148 1548 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d1c609779669456d8e14b87f86457ee3 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:59:15.212420 1548 raft_consensus.cc:686] T 00000000000000000000000000000000 P d1c609779669456d8e14b87f86457ee3 [term 1 LEADER]: Becoming Leader. State: Replica: d1c609779669456d8e14b87f86457ee3, State: Running, Role: LEADER
I20260811 18:59:15.212812 1548 consensus_queue.cc:237] T 00000000000000000000000000000000 P d1c609779669456d8e14b87f86457ee3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1c609779669456d8e14b87f86457ee3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46571 } }
I20260811 18:59:15.213177 1530 sys_catalog.cc:564] T 00000000000000000000000000000000 P d1c609779669456d8e14b87f86457ee3 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:59:15.214535 1551 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1c609779669456d8e14b87f86457ee3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d1c609779669456d8e14b87f86457ee3. Latest consensus state: current_term: 1 leader_uuid: "d1c609779669456d8e14b87f86457ee3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1c609779669456d8e14b87f86457ee3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46571 } } }
I20260811 18:59:15.214658 1551 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1c609779669456d8e14b87f86457ee3 [sys.catalog]: This master's current role is: LEADER
I20260811 18:59:15.215047 1550 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1c609779669456d8e14b87f86457ee3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d1c609779669456d8e14b87f86457ee3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1c609779669456d8e14b87f86457ee3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46571 } } }
I20260811 18:59:15.215189 1550 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1c609779669456d8e14b87f86457ee3 [sys.catalog]: This master's current role is: LEADER
I20260811 18:59:15.217563 1547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:15.202550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41057 (local address 127.0.0.1:46571)
0811 18:59:15.202960 (+ 410us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:15.202976 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:15.203011 (+ 35us) server_negotiation.cc:407] Connection header received
0811 18:59:15.203866 (+ 855us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:15.203891 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:15.204301 (+ 410us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:15.204668 (+ 367us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:15.205492 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:15.207041 (+ 1549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:15.207936 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:15.208646 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:15.208813 (+ 167us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:15.211540 (+ 2727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:15.211545 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:15.211559 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:59:15.211586 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:15.213605 (+ 2019us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:15.214237 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:15.214244 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:15.214249 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:15.214411 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:15.214833 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:15.214837 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:15.214839 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:15.215378 (+ 539us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:15.215597 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:15.216374 (+ 777us) server_negotiation.cc:299] Negotiation successful
0811 18:59:15.216604 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":154,"threads_started":1}
W20260811 18:59:15.218339 1577 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d1c609779669456d8e14b87f86457ee3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:59:15.218415 1577 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:59:15.218489 1571 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:59:15.219336 1571 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:59:15.220908 28370 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:59:15.225067 1571 catalog_manager.cc:1347] Generated new cluster ID: 2de9361949674243a128330560abb545
I20260811 18:59:15.225152 1571 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:59:15.233325 1571 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:59:15.234983 1571 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:59:15.235708 1547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:15.222849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43805 (local address 127.0.0.1:46571)
0811 18:59:15.222952 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:15.222956 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:15.223185 (+ 229us) server_negotiation.cc:407] Connection header received
0811 18:59:15.223280 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:15.223283 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:15.223323 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:15.223390 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:15.224095 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:15.224881 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:15.230203 (+ 5322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:15.230572 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:15.230681 (+ 109us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:15.232203 (+ 1522us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:15.232207 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:15.232210 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:59:15.232236 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:15.233812 (+ 1576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:15.234334 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:15.234338 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:15.234340 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:15.234431 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:15.234773 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:15.234776 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:15.234778 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:15.234973 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:15.235112 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:15.235459 (+ 347us) server_negotiation.cc:299] Negotiation successful
0811 18:59:15.235559 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 18:59:15.243081 1571 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d1c609779669456d8e14b87f86457ee3: Generated new TSK 0
I20260811 18:59:15.243834 1571 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:59:15.247937 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 1410
W20260811 18:59:15.265471 1375 connection.cc:535] client connection to 127.0.0.1:46571 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46571): Connection reset by peer (error 104)
2026-08-11T18:59:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1576 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474496157106-28370-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:59:15 dist-test-slave-2j7r krb5kdc[1594](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:59:15 dist-test-slave-2j7r krb5kdc[1594](info): set up 2 sockets
Aug 11 18:59:15 dist-test-slave-2j7r krb5kdc[1594](info): commencing operation
krb5kdc: starting...
W20260811 18:59:16.451167 28370 mini_kdc.cc:121] Time spent starting KDC: real 1.172s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474496157106-28370-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474496157106-28370-0/krb5kdc/test-user.keytab.
Aug 11 18:59:16 dist-test-slave-2j7r krb5kdc[1594](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474756, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:59:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:59:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474496157106-28370-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474496157106-28370-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 18:59:16.580828 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474496157106-28370-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474496157106-28370-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474496157106-28370-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35557
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474496157106-28370-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35557
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:59:16.693214 1628 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:59:16.693495 1628 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:59:16.693554 1628 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:59:16.697885 1628 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:59:16.697979 1628 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:59:16.698017 1628 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:59:16.698043 1628 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:59:16.698079 1628 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:59:16.703835 1628 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474496157106-28370-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474496157106-28370-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35557
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474496157106-28370-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35557
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1628
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474496157106-28370-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:59:16.705032 1628 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:59:16.706748 1628 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:59:16.715610 1634 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:16.715646 1638 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:16.715622 1642 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:59:16.715716 1628 server_base.cc:1034] running on GCE node
I20260811 18:59:16.716492 1628 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:59:16.717710 1628 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:59:16.718896 1628 hybrid_clock.cc:648] HybridClock initialized: now 1786474756718871 us; error 45 us; skew 500 ppm
Aug 11 18:59:16 dist-test-slave-2j7r krb5kdc[1594](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474756, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:59:16.721709 1628 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 18:59:16.722679 1628 webserver.cc:460] Webserver started at http://127.0.0.1:39717/ using document root <none> and password file <none>
I20260811 18:59:16.723234 1628 fs_manager.cc:359] Metadata directory not provided
I20260811 18:59:16.723304 1628 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:59:16.723513 1628 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:59:16.725662 1628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474496157106-28370-0/minicluster-data/master-0/data/instance:
uuid: "dd4eb806059d4b67b44edab7346698ad"
format_stamp: "Formatted at 2026-08-11 18:59:16 on dist-test-slave-2j7r"
server_key: "4bcaa3f846ffa6370009fe145223b6e0"
server_key_iv: "ef8c34c9433785d88ab7727925294919"
server_key_version: "encryptionkey@0"
I20260811 18:59:16.726253 1628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474496157106-28370-0/minicluster-data/master-0/wal/instance:
uuid: "dd4eb806059d4b67b44edab7346698ad"
format_stamp: "Formatted at 2026-08-11 18:59:16 on dist-test-slave-2j7r"
server_key: "4bcaa3f846ffa6370009fe145223b6e0"
server_key_iv: "ef8c34c9433785d88ab7727925294919"
server_key_version: "encryptionkey@0"
I20260811 18:59:16.730208 1628 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260811 18:59:16.733031 1652 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:16.734372 1628 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 18:59:16.734499 1628 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474496157106-28370-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474496157106-28370-0/minicluster-data/master-0/wal
uuid: "dd4eb806059d4b67b44edab7346698ad"
format_stamp: "Formatted at 2026-08-11 18:59:16 on dist-test-slave-2j7r"
server_key: "4bcaa3f846ffa6370009fe145223b6e0"
server_key_iv: "ef8c34c9433785d88ab7727925294919"
server_key_version: "encryptionkey@0"
I20260811 18:59:16.734606 1628 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474496157106-28370-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474496157106-28370-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474496157106-28370-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:59:16.748272 1628 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:59:16.751475 1628 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:59:16.751662 1628 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:59:16.758839 1628 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35557
I20260811 18:59:16.758882 1773 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35557 every 8 connection(s)
I20260811 18:59:16.759864 1628 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
I20260811 18:59:16.762749 1774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:59:16.768342 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 1628
I20260811 18:59:16.768407 1774 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd4eb806059d4b67b44edab7346698ad: Bootstrap starting.
I20260811 18:59:16.768471 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786474496157106-28370-0/minicluster-data/master-0/wal/instance
I20260811 18:59:16.768738 28370 external_mini_cluster.cc:1428] Setting key 61e089d26cd58c1d2a23d43e78099cca
I20260811 18:59:16.771001 1774 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dd4eb806059d4b67b44edab7346698ad: Neither blocks nor log segments found. Creating new log.
I20260811 18:59:16.771718 1774 log.cc:824] T 00000000000000000000000000000000 P dd4eb806059d4b67b44edab7346698ad: Log is configured to *not* fsync() on all Append() calls
I20260811 18:59:16.773628 1774 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd4eb806059d4b67b44edab7346698ad: No bootstrap required, opened a new log
I20260811 18:59:16.777165 1774 raft_consensus.cc:348] T 00000000000000000000000000000000 P dd4eb806059d4b67b44edab7346698ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd4eb806059d4b67b44edab7346698ad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35557 } }
I20260811 18:59:16.777390 1774 raft_consensus.cc:374] T 00000000000000000000000000000000 P dd4eb806059d4b67b44edab7346698ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:59:16.777400 1778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:16.770098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53915 (local address 127.0.0.1:35557)
0811 18:59:16.770541 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:16.770556 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:16.770590 (+ 34us) server_negotiation.cc:407] Connection header received
0811 18:59:16.771231 (+ 641us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:16.771250 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:16.771634 (+ 384us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:16.771953 (+ 319us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:16.772365 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:16.773363 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:16.774253 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:16.774719 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:16.774878 (+ 159us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:16.775696 (+ 818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:16.775701 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:16.775715 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:59:16.776102 (+ 387us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:16.776154 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:59:16.776177 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:16.776304 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:16.776476 (+ 172us) server_negotiation.cc:299] Negotiation successful
0811 18:59:16.776609 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":124,"threads_started":1}
I20260811 18:59:16.777457 1774 raft_consensus.cc:729] T 00000000000000000000000000000000 P dd4eb806059d4b67b44edab7346698ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd4eb806059d4b67b44edab7346698ad, State: Initialized, Role: FOLLOWER
I20260811 18:59:16.777940 1774 consensus_queue.cc:260] T 00000000000000000000000000000000 P dd4eb806059d4b67b44edab7346698ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd4eb806059d4b67b44edab7346698ad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35557 } }
I20260811 18:59:16.778103 1774 raft_consensus.cc:388] T 00000000000000000000000000000000 P dd4eb806059d4b67b44edab7346698ad [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:59:16.778165 1774 raft_consensus.cc:482] T 00000000000000000000000000000000 P dd4eb806059d4b67b44edab7346698ad [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:59:16.778258 1774 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dd4eb806059d4b67b44edab7346698ad [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:59:16.779237 1774 raft_consensus.cc:504] T 00000000000000000000000000000000 P dd4eb806059d4b67b44edab7346698ad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd4eb806059d4b67b44edab7346698ad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35557 } }
I20260811 18:59:16.779578 1774 leader_election.cc:302] T 00000000000000000000000000000000 P dd4eb806059d4b67b44edab7346698ad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dd4eb806059d4b67b44edab7346698ad; no voters:
I20260811 18:59:16.779878 1774 leader_election.cc:288] T 00000000000000000000000000000000 P dd4eb806059d4b67b44edab7346698ad [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:59:16.779965 1780 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dd4eb806059d4b67b44edab7346698ad [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:59:16.780185 1780 raft_consensus.cc:686] T 00000000000000000000000000000000 P dd4eb806059d4b67b44edab7346698ad [term 1 LEADER]: Becoming Leader. State: Replica: dd4eb806059d4b67b44edab7346698ad, State: Running, Role: LEADER
I20260811 18:59:16.780356 28370 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:59:16.780560 1780 consensus_queue.cc:237] T 00000000000000000000000000000000 P dd4eb806059d4b67b44edab7346698ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd4eb806059d4b67b44edab7346698ad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35557 } }
I20260811 18:59:16.781237 1774 sys_catalog.cc:564] T 00000000000000000000000000000000 P dd4eb806059d4b67b44edab7346698ad [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:59:16.781839 1781 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd4eb806059d4b67b44edab7346698ad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dd4eb806059d4b67b44edab7346698ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd4eb806059d4b67b44edab7346698ad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35557 } } }
I20260811 18:59:16.781845 1780 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd4eb806059d4b67b44edab7346698ad [sys.catalog]: SysCatalogTable state changed. Reason: New leader dd4eb806059d4b67b44edab7346698ad. Latest consensus state: current_term: 1 leader_uuid: "dd4eb806059d4b67b44edab7346698ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd4eb806059d4b67b44edab7346698ad" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35557 } } }
I20260811 18:59:16.781991 1780 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd4eb806059d4b67b44edab7346698ad [sys.catalog]: This master's current role is: LEADER
I20260811 18:59:16.781991 1781 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd4eb806059d4b67b44edab7346698ad [sys.catalog]: This master's current role is: LEADER
I20260811 18:59:16.782586 1798 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:59:16.786505 1798 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:59:16.787786 1778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:16.782506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46193 (local address 127.0.0.1:35557)
0811 18:59:16.782609 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:16.782613 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:16.783184 (+ 571us) server_negotiation.cc:407] Connection header received
0811 18:59:16.783379 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:16.783383 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:16.783432 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:16.783481 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:16.783932 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:16.784503 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:16.785677 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:16.785949 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:16.786037 (+ 88us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:16.787211 (+ 1174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:16.787215 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:16.787218 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:59:16.787233 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:16.787263 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:59:16.787290 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:16.787356 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:16.787605 (+ 249us) server_negotiation.cc:299] Negotiation successful
0811 18:59:16.787645 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260811 18:59:16.791965 1798 catalog_manager.cc:1347] Generated new cluster ID: fc48f1989ede408faeedb48f82e558e1
I20260811 18:59:16.792032 1798 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:59:16.805778 1798 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:59:16.807036 1798 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:59:16.812422 1798 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dd4eb806059d4b67b44edab7346698ad: Generated new TSK 0
I20260811 18:59:16.813196 1798 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:59:16.823107 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 1628
W20260811 18:59:16.832773 1592 connection.cc:535] client connection to 127.0.0.1:35557 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35557): Connection reset by peer (error 104)
2026-08-11T18:59:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1567 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474496157106-28370-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:59:16 dist-test-slave-2j7r krb5kdc[1827](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:59:16 dist-test-slave-2j7r krb5kdc[1827](info): set up 2 sockets
Aug 11 18:59:16 dist-test-slave-2j7r krb5kdc[1827](info): commencing operation
krb5kdc: starting...
W20260811 18:59:18.028849 28370 mini_kdc.cc:121] Time spent starting KDC: real 1.182s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474496157106-28370-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474496157106-28370-0/krb5kdc/test-user.keytab.
Aug 11 18:59:18 dist-test-slave-2j7r krb5kdc[1827](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474758, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:59:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:59:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474496157106-28370-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474496157106-28370-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 18:59:18.147351 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474496157106-28370-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474496157106-28370-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474496157106-28370-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33593
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474496157106-28370-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39361
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33593
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:59:18.257905 1849 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:59:18.258159 1849 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:59:18.258200 1849 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:59:18.262156 1849 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:59:18.262216 1849 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:59:18.262235 1849 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:59:18.262249 1849 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:59:18.262264 1849 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:59:18.267499 1849 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39361
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474496157106-28370-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474496157106-28370-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33593
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474496157106-28370-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33593
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1849
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474496157106-28370-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:59:18.268492 1849 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:59:18.269491 1849 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:59:18.277729 1864 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:18.277700 1858 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:18.277915 1859 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:59:18.277912 1849 server_base.cc:1034] running on GCE node
I20260811 18:59:18.278535 1849 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:59:18.279631 1849 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:59:18.280791 1849 hybrid_clock.cc:648] HybridClock initialized: now 1786474758280774 us; error 30 us; skew 500 ppm
Aug 11 18:59:18 dist-test-slave-2j7r krb5kdc[1827](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474758, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:59:18.288270 1849 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 18:59:18.289247 1849 webserver.cc:460] Webserver started at http://127.0.0.1:42137/ using document root <none> and password file <none>
I20260811 18:59:18.289788 1849 fs_manager.cc:359] Metadata directory not provided
I20260811 18:59:18.289849 1849 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:59:18.290025 1849 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:59:18.292104 1849 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474496157106-28370-0/minicluster-data/master-0/data/instance:
uuid: "1783896b45c94c3aa371925bd855a48f"
format_stamp: "Formatted at 2026-08-11 18:59:18 on dist-test-slave-2j7r"
server_key: "46c77d0ebaa814ebeca1588e1204fd35"
server_key_iv: "5984c857b2759255b79fb52b0519a79d"
server_key_version: "encryptionkey@0"
I20260811 18:59:18.292641 1849 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474496157106-28370-0/minicluster-data/master-0/wal/instance:
uuid: "1783896b45c94c3aa371925bd855a48f"
format_stamp: "Formatted at 2026-08-11 18:59:18 on dist-test-slave-2j7r"
server_key: "46c77d0ebaa814ebeca1588e1204fd35"
server_key_iv: "5984c857b2759255b79fb52b0519a79d"
server_key_version: "encryptionkey@0"
I20260811 18:59:18.296454 1849 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 18:59:18.298877 1876 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:18.300107 1849 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260811 18:59:18.300220 1849 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474496157106-28370-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474496157106-28370-0/minicluster-data/master-0/wal
uuid: "1783896b45c94c3aa371925bd855a48f"
format_stamp: "Formatted at 2026-08-11 18:59:18 on dist-test-slave-2j7r"
server_key: "46c77d0ebaa814ebeca1588e1204fd35"
server_key_iv: "5984c857b2759255b79fb52b0519a79d"
server_key_version: "encryptionkey@0"
I20260811 18:59:18.300326 1849 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474496157106-28370-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474496157106-28370-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474496157106-28370-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:59:18.305243 1849 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:59:18.308823 1849 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:59:18.309008 1849 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:59:18.316015 1849 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33593
I20260811 18:59:18.316030 1972 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33593 every 8 connection(s)
I20260811 18:59:18.316923 1849 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
I20260811 18:59:18.320386 1973 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:59:18.323786 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 1849
I20260811 18:59:18.323930 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786474496157106-28370-0/minicluster-data/master-0/wal/instance
I20260811 18:59:18.324187 28370 external_mini_cluster.cc:1428] Setting key 6ced572490823ec1c68b72a4382ed71f
I20260811 18:59:18.326138 1973 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1783896b45c94c3aa371925bd855a48f: Bootstrap starting.
I20260811 18:59:18.328317 1973 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1783896b45c94c3aa371925bd855a48f: Neither blocks nor log segments found. Creating new log.
I20260811 18:59:18.329103 1973 log.cc:824] T 00000000000000000000000000000000 P 1783896b45c94c3aa371925bd855a48f: Log is configured to *not* fsync() on all Append() calls
I20260811 18:59:18.329303 1982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:18.325568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41209 (local address 127.0.0.1:33593)
0811 18:59:18.326010 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:18.326025 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:18.326061 (+ 36us) server_negotiation.cc:407] Connection header received
0811 18:59:18.326789 (+ 728us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:18.326809 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:18.327084 (+ 275us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:18.327391 (+ 307us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:18.328110 (+ 719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:18.328115 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:18.328126 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:59:18.328136 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:18.328193 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:59:18.328211 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:18.328304 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:18.328420 (+ 116us) server_negotiation.cc:299] Negotiation successful
0811 18:59:18.328571 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285,"thread_start_us":129,"threads_started":1}
I20260811 18:59:18.330978 1973 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1783896b45c94c3aa371925bd855a48f: No bootstrap required, opened a new log
I20260811 18:59:18.332041 28370 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:59:18.334712 1973 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1783896b45c94c3aa371925bd855a48f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1783896b45c94c3aa371925bd855a48f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33593 } }
I20260811 18:59:18.334937 1973 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1783896b45c94c3aa371925bd855a48f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:59:18.335004 1973 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1783896b45c94c3aa371925bd855a48f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1783896b45c94c3aa371925bd855a48f, State: Initialized, Role: FOLLOWER
I20260811 18:59:18.335450 1973 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1783896b45c94c3aa371925bd855a48f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1783896b45c94c3aa371925bd855a48f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33593 } }
I20260811 18:59:18.335594 1982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:18.334114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41583 (local address 127.0.0.1:33593)
0811 18:59:18.334228 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:18.334232 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:18.334383 (+ 151us) server_negotiation.cc:407] Connection header received
0811 18:59:18.334473 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:18.334476 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:18.334509 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:18.334583 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:18.335245 (+ 662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:18.335249 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:18.335251 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:59:18.335263 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:18.335289 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:59:18.335302 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:18.335395 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:18.335457 (+ 62us) server_negotiation.cc:299] Negotiation successful
0811 18:59:18.335493 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 18:59:18.335649 1973 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1783896b45c94c3aa371925bd855a48f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:59:18.335712 1973 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1783896b45c94c3aa371925bd855a48f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:59:18.335816 1973 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1783896b45c94c3aa371925bd855a48f [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:59:18.336769 1973 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1783896b45c94c3aa371925bd855a48f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1783896b45c94c3aa371925bd855a48f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33593 } }
I20260811 18:59:18.337111 1973 leader_election.cc:302] T 00000000000000000000000000000000 P 1783896b45c94c3aa371925bd855a48f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1783896b45c94c3aa371925bd855a48f; no voters:
I20260811 18:59:18.337401 1973 leader_election.cc:288] T 00000000000000000000000000000000 P 1783896b45c94c3aa371925bd855a48f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:59:18.337520 1998 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1783896b45c94c3aa371925bd855a48f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:59:18.337723 1998 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1783896b45c94c3aa371925bd855a48f [term 1 LEADER]: Becoming Leader. State: Replica: 1783896b45c94c3aa371925bd855a48f, State: Running, Role: LEADER
I20260811 18:59:18.338063 1998 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1783896b45c94c3aa371925bd855a48f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1783896b45c94c3aa371925bd855a48f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33593 } }
I20260811 18:59:18.338495 1973 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1783896b45c94c3aa371925bd855a48f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:59:18.339810 1999 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1783896b45c94c3aa371925bd855a48f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1783896b45c94c3aa371925bd855a48f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1783896b45c94c3aa371925bd855a48f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33593 } } }
I20260811 18:59:18.339937 1999 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1783896b45c94c3aa371925bd855a48f [sys.catalog]: This master's current role is: LEADER
I20260811 18:59:18.340483 2000 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1783896b45c94c3aa371925bd855a48f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1783896b45c94c3aa371925bd855a48f. Latest consensus state: current_term: 1 leader_uuid: "1783896b45c94c3aa371925bd855a48f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1783896b45c94c3aa371925bd855a48f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33593 } } }
I20260811 18:59:18.340615 2018 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:59:18.340870 2000 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1783896b45c94c3aa371925bd855a48f [sys.catalog]: This master's current role is: LEADER
I20260811 18:59:18.343554 2018 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:59:18.348744 2018 catalog_manager.cc:1347] Generated new cluster ID: f31410d324f24bf6961d1f65fd934098
I20260811 18:59:18.348810 2018 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:59:18.364169 2018 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:59:18.364477 2018 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:59:18.369886 2018 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1783896b45c94c3aa371925bd855a48f: Generated new TSK 0
I20260811 18:59:18.370575 2018 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:59:18.383528 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 1849
2026-08-11T18:59:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1570 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474496157106-28370-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:59:18 dist-test-slave-2j7r krb5kdc[2034](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:59:18 dist-test-slave-2j7r krb5kdc[2034](info): set up 2 sockets
Aug 11 18:59:18 dist-test-slave-2j7r krb5kdc[2034](info): commencing operation
krb5kdc: starting...
W20260811 18:59:19.588577 28370 mini_kdc.cc:121] Time spent starting KDC: real 1.171s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474496157106-28370-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474496157106-28370-0/krb5kdc/test-user.keytab.
Aug 11 18:59:19 dist-test-slave-2j7r krb5kdc[2034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474759, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:59:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:59:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474496157106-28370-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474496157106-28370-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 18:59:19.706250 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474496157106-28370-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474496157106-28370-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474496157106-28370-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46609
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474496157106-28370-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46609
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:59:19.816963 2054 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:59:19.817267 2054 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:59:19.817344 2054 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:59:19.821300 2054 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:59:19.821360 2054 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:59:19.821393 2054 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:59:19.821427 2054 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:59:19.821460 2054 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:59:19.826741 2054 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474496157106-28370-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474496157106-28370-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46609
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474496157106-28370-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46609
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2054
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474496157106-28370-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:59:19.827718 2054 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:59:19.828706 2054 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:59:19.836844 2077 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:19.836875 2075 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:59:19.837067 2054 server_base.cc:1034] running on GCE node
W20260811 18:59:19.836879 2081 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:59:19.837706 2054 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:59:19.838734 2054 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:59:19.839884 2054 hybrid_clock.cc:648] HybridClock initialized: now 1786474759839869 us; error 36 us; skew 500 ppm
Aug 11 18:59:19 dist-test-slave-2j7r krb5kdc[2034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474759, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:59:19.842572 2054 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 18:59:19.843428 2054 webserver.cc:460] Webserver started at http://127.0.0.1:34753/ using document root <none> and password file <none>
I20260811 18:59:19.843940 2054 fs_manager.cc:359] Metadata directory not provided
I20260811 18:59:19.844007 2054 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:59:19.844225 2054 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:59:19.846274 2054 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474496157106-28370-0/minicluster-data/master-0/data/instance:
uuid: "680533ad5c014f4f91b1b43e32c8746b"
format_stamp: "Formatted at 2026-08-11 18:59:19 on dist-test-slave-2j7r"
server_key: "fbaeb45a145e5002331438cf0013ba38"
server_key_iv: "2c3d735b1e8dafbca5d83d4d2ac9e65f"
server_key_version: "encryptionkey@0"
I20260811 18:59:19.846819 2054 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474496157106-28370-0/minicluster-data/master-0/wal/instance:
uuid: "680533ad5c014f4f91b1b43e32c8746b"
format_stamp: "Formatted at 2026-08-11 18:59:19 on dist-test-slave-2j7r"
server_key: "fbaeb45a145e5002331438cf0013ba38"
server_key_iv: "2c3d735b1e8dafbca5d83d4d2ac9e65f"
server_key_version: "encryptionkey@0"
I20260811 18:59:19.850443 2054 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 18:59:19.852774 2091 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:19.853977 2054 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 18:59:19.854110 2054 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474496157106-28370-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474496157106-28370-0/minicluster-data/master-0/wal
uuid: "680533ad5c014f4f91b1b43e32c8746b"
format_stamp: "Formatted at 2026-08-11 18:59:19 on dist-test-slave-2j7r"
server_key: "fbaeb45a145e5002331438cf0013ba38"
server_key_iv: "2c3d735b1e8dafbca5d83d4d2ac9e65f"
server_key_version: "encryptionkey@0"
I20260811 18:59:19.854205 2054 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474496157106-28370-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474496157106-28370-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474496157106-28370-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:59:19.873657 2054 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:59:19.877334 2054 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:59:19.877519 2054 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:59:19.885964 2054 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46609
I20260811 18:59:19.885988 2185 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46609 every 8 connection(s)
I20260811 18:59:19.887246 2054 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
I20260811 18:59:19.891615 2188 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:59:19.892050 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 2054
I20260811 18:59:19.892189 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786474496157106-28370-0/minicluster-data/master-0/wal/instance
I20260811 18:59:19.892520 28370 external_mini_cluster.cc:1428] Setting key d1849e703e747a28193e12e52a399012
I20260811 18:59:19.899098 2188 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 680533ad5c014f4f91b1b43e32c8746b: Bootstrap starting.
Aug 11 18:59:19 dist-test-slave-2j7r krb5kdc[2034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474759, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 18:59:19.901484 2188 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 680533ad5c014f4f91b1b43e32c8746b: Neither blocks nor log segments found. Creating new log.
I20260811 18:59:19.902323 2188 log.cc:824] T 00000000000000000000000000000000 P 680533ad5c014f4f91b1b43e32c8746b: Log is configured to *not* fsync() on all Append() calls
I20260811 18:59:19.904332 2188 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 680533ad5c014f4f91b1b43e32c8746b: No bootstrap required, opened a new log
I20260811 18:59:19.906436 2191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:19.893891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36519 (local address 127.0.0.1:46609)
0811 18:59:19.894354 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:19.894374 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:19.894422 (+ 48us) server_negotiation.cc:407] Connection header received
0811 18:59:19.895493 (+ 1071us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:19.895511 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:19.895837 (+ 326us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:19.896142 (+ 305us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:19.896955 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:19.898004 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:19.898844 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:19.899362 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:19.899510 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:19.902126 (+ 2616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:19.902131 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:19.902143 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:59:19.902171 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:19.903735 (+ 1564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:19.904197 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:19.904201 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:19.904205 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:19.904310 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:19.904588 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:19.904590 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:19.904591 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:19.904952 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:19.905118 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:19.905429 (+ 311us) server_negotiation.cc:299] Negotiation successful
0811 18:59:19.905708 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302,"thread_start_us":142,"threads_started":1}
I20260811 18:59:19.907989 2188 raft_consensus.cc:348] T 00000000000000000000000000000000 P 680533ad5c014f4f91b1b43e32c8746b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "680533ad5c014f4f91b1b43e32c8746b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46609 } }
I20260811 18:59:19.908219 2188 raft_consensus.cc:374] T 00000000000000000000000000000000 P 680533ad5c014f4f91b1b43e32c8746b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:59:19.908290 2188 raft_consensus.cc:729] T 00000000000000000000000000000000 P 680533ad5c014f4f91b1b43e32c8746b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 680533ad5c014f4f91b1b43e32c8746b, State: Initialized, Role: FOLLOWER
I20260811 18:59:19.908710 2188 consensus_queue.cc:260] T 00000000000000000000000000000000 P 680533ad5c014f4f91b1b43e32c8746b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "680533ad5c014f4f91b1b43e32c8746b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46609 } }
I20260811 18:59:19.908886 2188 raft_consensus.cc:388] T 00000000000000000000000000000000 P 680533ad5c014f4f91b1b43e32c8746b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:59:19.908994 2188 raft_consensus.cc:482] T 00000000000000000000000000000000 P 680533ad5c014f4f91b1b43e32c8746b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:59:19.909114 2188 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 680533ad5c014f4f91b1b43e32c8746b [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:59:19.909133 28370 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:59:19.909947 2188 raft_consensus.cc:504] T 00000000000000000000000000000000 P 680533ad5c014f4f91b1b43e32c8746b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "680533ad5c014f4f91b1b43e32c8746b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46609 } }
I20260811 18:59:19.910291 2188 leader_election.cc:302] T 00000000000000000000000000000000 P 680533ad5c014f4f91b1b43e32c8746b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 680533ad5c014f4f91b1b43e32c8746b; no voters:
I20260811 18:59:19.910589 2188 leader_election.cc:288] T 00000000000000000000000000000000 P 680533ad5c014f4f91b1b43e32c8746b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:59:19.910707 2200 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 680533ad5c014f4f91b1b43e32c8746b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:59:19.911012 2200 raft_consensus.cc:686] T 00000000000000000000000000000000 P 680533ad5c014f4f91b1b43e32c8746b [term 1 LEADER]: Becoming Leader. State: Replica: 680533ad5c014f4f91b1b43e32c8746b, State: Running, Role: LEADER
I20260811 18:59:19.911322 2200 consensus_queue.cc:237] T 00000000000000000000000000000000 P 680533ad5c014f4f91b1b43e32c8746b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "680533ad5c014f4f91b1b43e32c8746b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46609 } }
I20260811 18:59:19.911855 2188 sys_catalog.cc:564] T 00000000000000000000000000000000 P 680533ad5c014f4f91b1b43e32c8746b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:59:19.912988 2202 sys_catalog.cc:455] T 00000000000000000000000000000000 P 680533ad5c014f4f91b1b43e32c8746b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "680533ad5c014f4f91b1b43e32c8746b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "680533ad5c014f4f91b1b43e32c8746b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46609 } } }
I20260811 18:59:19.913192 2202 sys_catalog.cc:458] T 00000000000000000000000000000000 P 680533ad5c014f4f91b1b43e32c8746b [sys.catalog]: This master's current role is: LEADER
I20260811 18:59:19.913446 2202 sys_catalog.cc:455] T 00000000000000000000000000000000 P 680533ad5c014f4f91b1b43e32c8746b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 680533ad5c014f4f91b1b43e32c8746b. Latest consensus state: current_term: 1 leader_uuid: "680533ad5c014f4f91b1b43e32c8746b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "680533ad5c014f4f91b1b43e32c8746b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46609 } } }
I20260811 18:59:19.913527 2202 sys_catalog.cc:458] T 00000000000000000000000000000000 P 680533ad5c014f4f91b1b43e32c8746b [sys.catalog]: This master's current role is: LEADER
W20260811 18:59:19.916972 2242 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 680533ad5c014f4f91b1b43e32c8746b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:59:19.917073 2242 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:59:19.917462 2234 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:59:19.918432 2234 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:59:19.919186 2191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:19.911387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58783 (local address 127.0.0.1:46609)
0811 18:59:19.911484 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:19.911488 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:19.911502 (+ 14us) server_negotiation.cc:407] Connection header received
0811 18:59:19.911536 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:19.911539 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:19.911587 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:19.911665 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:19.912528 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:19.913339 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:19.914454 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:19.914671 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:19.914764 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:19.915983 (+ 1219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:19.915986 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:19.915988 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:59:19.916009 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:19.917398 (+ 1389us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:19.917848 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:19.917852 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:19.917854 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:19.917946 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:19.918311 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:19.918315 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:19.918316 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:19.918540 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:19.918673 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:19.918920 (+ 247us) server_negotiation.cc:299] Negotiation successful
0811 18:59:19.919037 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260811 18:59:19.923656 2234 catalog_manager.cc:1347] Generated new cluster ID: b1e5904684b34f3cad9a984c75a3b252
I20260811 18:59:19.923709 2234 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:59:19.931506 2234 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:59:19.932380 2234 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:59:19.942492 2234 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 680533ad5c014f4f91b1b43e32c8746b: Generated new TSK 0
I20260811 18:59:19.943094 2234 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:59:19.955319 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 2054
2026-08-11T18:59:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1563 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474496157106-28370-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:59:20 dist-test-slave-2j7r krb5kdc[2251](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:59:20 dist-test-slave-2j7r krb5kdc[2251](info): set up 2 sockets
Aug 11 18:59:20 dist-test-slave-2j7r krb5kdc[2251](info): commencing operation
krb5kdc: starting...
W20260811 18:59:21.146217 28370 mini_kdc.cc:121] Time spent starting KDC: real 1.165s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474496157106-28370-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474496157106-28370-0/krb5kdc/test-user.keytab.
Aug 11 18:59:21 dist-test-slave-2j7r krb5kdc[2251](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474761, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:59:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:59:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474496157106-28370-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474496157106-28370-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 18:59:21.265841 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474496157106-28370-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474496157106-28370-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474496157106-28370-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41063
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474496157106-28370-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41063
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:59:21.375133 2283 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:59:21.375366 2283 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:59:21.375422 2283 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:59:21.379294 2283 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:59:21.379347 2283 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:59:21.379372 2283 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:59:21.379395 2283 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:59:21.379422 2283 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:59:21.384476 2283 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474496157106-28370-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474496157106-28370-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41063
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474496157106-28370-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41063
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2283
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474496157106-28370-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:59:21.385453 2283 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:59:21.386408 2283 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:59:21.394516 2298 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:21.394480 2302 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:21.394484 2297 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:59:21.394884 2283 server_base.cc:1034] running on GCE node
I20260811 18:59:21.395396 2283 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:59:21.396385 2283 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:59:21.397541 2283 hybrid_clock.cc:648] HybridClock initialized: now 1786474761397523 us; error 29 us; skew 500 ppm
Aug 11 18:59:21 dist-test-slave-2j7r krb5kdc[2251](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474761, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:59:21.400223 2283 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 18:59:21.401060 2283 webserver.cc:460] Webserver started at http://127.0.0.1:40509/ using document root <none> and password file <none>
I20260811 18:59:21.401597 2283 fs_manager.cc:359] Metadata directory not provided
I20260811 18:59:21.401661 2283 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:59:21.401867 2283 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:59:21.403843 2283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474496157106-28370-0/minicluster-data/master-0/data/instance:
uuid: "5d02fce9c05d4f3384ff0da589355839"
format_stamp: "Formatted at 2026-08-11 18:59:21 on dist-test-slave-2j7r"
server_key: "e20291c2b873f5bb1bbdf773ea1bc0b4"
server_key_iv: "e16c88286c5f8506c9a998f06843af75"
server_key_version: "encryptionkey@0"
I20260811 18:59:21.404366 2283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474496157106-28370-0/minicluster-data/master-0/wal/instance:
uuid: "5d02fce9c05d4f3384ff0da589355839"
format_stamp: "Formatted at 2026-08-11 18:59:21 on dist-test-slave-2j7r"
server_key: "e20291c2b873f5bb1bbdf773ea1bc0b4"
server_key_iv: "e16c88286c5f8506c9a998f06843af75"
server_key_version: "encryptionkey@0"
I20260811 18:59:21.408025 2283 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260811 18:59:21.410501 2311 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:21.411721 2283 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 18:59:21.411870 2283 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474496157106-28370-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474496157106-28370-0/minicluster-data/master-0/wal
uuid: "5d02fce9c05d4f3384ff0da589355839"
format_stamp: "Formatted at 2026-08-11 18:59:21 on dist-test-slave-2j7r"
server_key: "e20291c2b873f5bb1bbdf773ea1bc0b4"
server_key_iv: "e16c88286c5f8506c9a998f06843af75"
server_key_version: "encryptionkey@0"
I20260811 18:59:21.411968 2283 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474496157106-28370-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474496157106-28370-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474496157106-28370-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:59:21.431383 2283 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:59:21.434527 2283 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:59:21.434710 2283 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:59:21.441612 2283 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41063
I20260811 18:59:21.441658 2404 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41063 every 8 connection(s)
I20260811 18:59:21.442600 2283 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
I20260811 18:59:21.445509 2406 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:59:21.450942 2406 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d02fce9c05d4f3384ff0da589355839: Bootstrap starting.
I20260811 18:59:21.451900 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 2283
I20260811 18:59:21.452037 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786474496157106-28370-0/minicluster-data/master-0/wal/instance
I20260811 18:59:21.452345 28370 external_mini_cluster.cc:1428] Setting key c828bbe89259df913197dd59c031ea9e
I20260811 18:59:21.453804 2406 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5d02fce9c05d4f3384ff0da589355839: Neither blocks nor log segments found. Creating new log.
I20260811 18:59:21.454495 2406 log.cc:824] T 00000000000000000000000000000000 P 5d02fce9c05d4f3384ff0da589355839: Log is configured to *not* fsync() on all Append() calls
I20260811 18:59:21.456303 2406 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d02fce9c05d4f3384ff0da589355839: No bootstrap required, opened a new log
I20260811 18:59:21.459686 2406 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5d02fce9c05d4f3384ff0da589355839 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d02fce9c05d4f3384ff0da589355839" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41063 } }
I20260811 18:59:21.459934 2406 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5d02fce9c05d4f3384ff0da589355839 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:59:21.460021 2406 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5d02fce9c05d4f3384ff0da589355839 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d02fce9c05d4f3384ff0da589355839, State: Initialized, Role: FOLLOWER
I20260811 18:59:21.460420 2406 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5d02fce9c05d4f3384ff0da589355839 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d02fce9c05d4f3384ff0da589355839" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41063 } }
I20260811 18:59:21.460549 2406 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5d02fce9c05d4f3384ff0da589355839 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:59:21.460613 2406 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5d02fce9c05d4f3384ff0da589355839 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:59:21.460711 2406 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5d02fce9c05d4f3384ff0da589355839 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:59:21.460832 2413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:21.453479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55469 (local address 127.0.0.1:41063)
0811 18:59:21.453891 (+ 412us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:21.453911 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:21.453950 (+ 39us) server_negotiation.cc:407] Connection header received
0811 18:59:21.454581 (+ 631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:21.454602 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:21.454952 (+ 350us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:21.455252 (+ 300us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:21.455650 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:21.456658 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:21.457595 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:21.458071 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:21.458227 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:21.459043 (+ 816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:21.459048 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:21.459059 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:59:21.459443 (+ 384us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:21.459498 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:59:21.459521 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:21.459606 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:21.459748 (+ 142us) server_negotiation.cc:299] Negotiation successful
0811 18:59:21.459963 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":255,"thread_start_us":96,"threads_started":1}
I20260811 18:59:21.461735 2406 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5d02fce9c05d4f3384ff0da589355839 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d02fce9c05d4f3384ff0da589355839" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41063 } }
I20260811 18:59:21.462065 2406 leader_election.cc:302] T 00000000000000000000000000000000 P 5d02fce9c05d4f3384ff0da589355839 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5d02fce9c05d4f3384ff0da589355839; no voters:
I20260811 18:59:21.462328 2406 leader_election.cc:288] T 00000000000000000000000000000000 P 5d02fce9c05d4f3384ff0da589355839 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:59:21.462421 2418 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5d02fce9c05d4f3384ff0da589355839 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:59:21.462628 2418 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5d02fce9c05d4f3384ff0da589355839 [term 1 LEADER]: Becoming Leader. State: Replica: 5d02fce9c05d4f3384ff0da589355839, State: Running, Role: LEADER
I20260811 18:59:21.462980 2418 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5d02fce9c05d4f3384ff0da589355839 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d02fce9c05d4f3384ff0da589355839" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41063 } }
I20260811 18:59:21.463574 2406 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5d02fce9c05d4f3384ff0da589355839 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:59:21.463838 28370 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:59:21.464533 2420 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d02fce9c05d4f3384ff0da589355839 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5d02fce9c05d4f3384ff0da589355839. Latest consensus state: current_term: 1 leader_uuid: "5d02fce9c05d4f3384ff0da589355839" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d02fce9c05d4f3384ff0da589355839" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41063 } } }
I20260811 18:59:21.464654 2420 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d02fce9c05d4f3384ff0da589355839 [sys.catalog]: This master's current role is: LEADER
I20260811 18:59:21.465903 2419 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d02fce9c05d4f3384ff0da589355839 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5d02fce9c05d4f3384ff0da589355839" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d02fce9c05d4f3384ff0da589355839" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41063 } } }
I20260811 18:59:21.466388 2419 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d02fce9c05d4f3384ff0da589355839 [sys.catalog]: This master's current role is: LEADER
W20260811 18:59:21.468739 2449 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5d02fce9c05d4f3384ff0da589355839: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:59:21.468830 2449 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:59:21.468904 2435 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:59:21.469638 2435 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:59:21.470779 2413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:21.466299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45279 (local address 127.0.0.1:41063)
0811 18:59:21.466392 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:21.466396 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:21.466744 (+ 348us) server_negotiation.cc:407] Connection header received
0811 18:59:21.466912 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:21.466915 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:21.466961 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:21.467023 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:21.467290 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:21.468010 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:21.469105 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:21.469439 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:21.469549 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:21.470353 (+ 804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:21.470357 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:21.470359 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:59:21.470372 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:21.470400 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:59:21.470414 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:21.470491 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:21.470586 (+ 95us) server_negotiation.cc:299] Negotiation successful
0811 18:59:21.470640 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260811 18:59:21.474761 2435 catalog_manager.cc:1347] Generated new cluster ID: 8739020b5a114c22b82248f2dbc45e55
I20260811 18:59:21.474821 2435 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:59:21.492317 2435 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:59:21.493599 2435 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:59:21.504760 2435 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5d02fce9c05d4f3384ff0da589355839: Generated new TSK 0
I20260811 18:59:21.505586 2435 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:59:21.514773 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 2283
2026-08-11T18:59:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1569 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474496157106-28370-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:59:21 dist-test-slave-2j7r krb5kdc[2457](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:59:21 dist-test-slave-2j7r krb5kdc[2457](info): set up 2 sockets
Aug 11 18:59:21 dist-test-slave-2j7r krb5kdc[2457](info): commencing operation
krb5kdc: starting...
W20260811 18:59:22.715296 28370 mini_kdc.cc:121] Time spent starting KDC: real 1.165s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474496157106-28370-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474496157106-28370-0/krb5kdc/test-user.keytab.
Aug 11 18:59:22 dist-test-slave-2j7r krb5kdc[2457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474762, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:59:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:59:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474496157106-28370-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474496157106-28370-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 18:59:22.830044 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474496157106-28370-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474496157106-28370-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474496157106-28370-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36027
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474496157106-28370-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36027
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:59:22.938314 2482 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:59:22.938551 2482 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:59:22.938607 2482 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:59:22.942472 2482 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:59:22.942525 2482 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:59:22.942551 2482 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:59:22.942580 2482 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:59:22.942610 2482 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:59:22.947769 2482 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474496157106-28370-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474496157106-28370-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36027
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474496157106-28370-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36027
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2482
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474496157106-28370-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:59:22.948733 2482 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:59:22.949729 2482 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:59:22.957965 2494 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:59:22.957957 2482 server_base.cc:1034] running on GCE node
W20260811 18:59:22.957919 2490 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:22.957922 2498 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:59:22.958720 2482 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:59:22.959749 2482 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:59:22.960935 2482 hybrid_clock.cc:648] HybridClock initialized: now 1786474762960922 us; error 25 us; skew 500 ppm
Aug 11 18:59:22 dist-test-slave-2j7r krb5kdc[2457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474762, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:59:22.963579 2482 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 18:59:22.964425 2482 webserver.cc:460] Webserver started at http://127.0.0.1:39049/ using document root <none> and password file <none>
I20260811 18:59:22.964946 2482 fs_manager.cc:359] Metadata directory not provided
I20260811 18:59:22.965005 2482 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:59:22.965243 2482 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:59:22.967303 2482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474496157106-28370-0/minicluster-data/master-0/data/instance:
uuid: "7b3fdbcf97b849e591a68eac36be89bc"
format_stamp: "Formatted at 2026-08-11 18:59:22 on dist-test-slave-2j7r"
server_key: "da0598019a012c46ccc9771fd1a5e673"
server_key_iv: "c57ee86c500875fe0980802c23d1e547"
server_key_version: "encryptionkey@0"
I20260811 18:59:22.967851 2482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474496157106-28370-0/minicluster-data/master-0/wal/instance:
uuid: "7b3fdbcf97b849e591a68eac36be89bc"
format_stamp: "Formatted at 2026-08-11 18:59:22 on dist-test-slave-2j7r"
server_key: "da0598019a012c46ccc9771fd1a5e673"
server_key_iv: "c57ee86c500875fe0980802c23d1e547"
server_key_version: "encryptionkey@0"
I20260811 18:59:22.971370 2482 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260811 18:59:22.973672 2508 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:22.974826 2482 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 18:59:22.974931 2482 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474496157106-28370-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474496157106-28370-0/minicluster-data/master-0/wal
uuid: "7b3fdbcf97b849e591a68eac36be89bc"
format_stamp: "Formatted at 2026-08-11 18:59:22 on dist-test-slave-2j7r"
server_key: "da0598019a012c46ccc9771fd1a5e673"
server_key_iv: "c57ee86c500875fe0980802c23d1e547"
server_key_version: "encryptionkey@0"
I20260811 18:59:22.975029 2482 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474496157106-28370-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474496157106-28370-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474496157106-28370-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:59:22.979420 2482 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:59:22.982067 2482 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:59:22.982219 2482 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:59:22.989140 2482 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36027
I20260811 18:59:22.989197 2610 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36027 every 8 connection(s)
I20260811 18:59:22.990012 2482 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
I20260811 18:59:22.992628 2612 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:59:22.995751 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 2482
I20260811 18:59:22.995891 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786474496157106-28370-0/minicluster-data/master-0/wal/instance
I20260811 18:59:22.996172 28370 external_mini_cluster.cc:1428] Setting key f02fb22bb02b066ce6e35d35fb8fcc59
I20260811 18:59:22.998313 2612 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b3fdbcf97b849e591a68eac36be89bc: Bootstrap starting.
I20260811 18:59:23.000717 2612 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7b3fdbcf97b849e591a68eac36be89bc: Neither blocks nor log segments found. Creating new log.
I20260811 18:59:23.001394 2615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:22.997489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51955 (local address 127.0.0.1:36027)
0811 18:59:22.997924 (+ 435us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:22.997939 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:22.997977 (+ 38us) server_negotiation.cc:407] Connection header received
0811 18:59:22.998658 (+ 681us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:22.998680 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:22.998960 (+ 280us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:22.999244 (+ 284us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:23.000084 (+ 840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:23.000089 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:23.000100 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:59:23.000112 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:23.000167 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:59:23.000197 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:23.000290 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:23.000415 (+ 125us) server_negotiation.cc:299] Negotiation successful
0811 18:59:23.000559 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":129,"threads_started":1}
I20260811 18:59:23.001497 2612 log.cc:824] T 00000000000000000000000000000000 P 7b3fdbcf97b849e591a68eac36be89bc: Log is configured to *not* fsync() on all Append() calls
I20260811 18:59:23.003295 2612 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b3fdbcf97b849e591a68eac36be89bc: No bootstrap required, opened a new log
I20260811 18:59:23.004005 28370 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:59:23.007790 2615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:23.006347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56559 (local address 127.0.0.1:36027)
0811 18:59:23.006462 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:23.006467 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:23.006615 (+ 148us) server_negotiation.cc:407] Connection header received
0811 18:59:23.006745 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:23.006747 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:23.006775 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:23.006830 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:23.007465 (+ 635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:23.007468 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:23.007470 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:59:23.007480 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:23.007501 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:59:23.007510 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:23.007573 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:23.007661 (+ 88us) server_negotiation.cc:299] Negotiation successful
0811 18:59:23.007691 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 18:59:23.007593 2612 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7b3fdbcf97b849e591a68eac36be89bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b3fdbcf97b849e591a68eac36be89bc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36027 } }
I20260811 18:59:23.007918 2612 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7b3fdbcf97b849e591a68eac36be89bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:59:23.007982 2612 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7b3fdbcf97b849e591a68eac36be89bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b3fdbcf97b849e591a68eac36be89bc, State: Initialized, Role: FOLLOWER
I20260811 18:59:23.008425 2612 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7b3fdbcf97b849e591a68eac36be89bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b3fdbcf97b849e591a68eac36be89bc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36027 } }
I20260811 18:59:23.008585 2612 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7b3fdbcf97b849e591a68eac36be89bc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:59:23.008656 2612 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b3fdbcf97b849e591a68eac36be89bc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:59:23.008752 2612 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7b3fdbcf97b849e591a68eac36be89bc [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:59:23.009653 2612 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b3fdbcf97b849e591a68eac36be89bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b3fdbcf97b849e591a68eac36be89bc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36027 } }
I20260811 18:59:23.009972 2612 leader_election.cc:302] T 00000000000000000000000000000000 P 7b3fdbcf97b849e591a68eac36be89bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7b3fdbcf97b849e591a68eac36be89bc; no voters:
I20260811 18:59:23.010237 2612 leader_election.cc:288] T 00000000000000000000000000000000 P 7b3fdbcf97b849e591a68eac36be89bc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:59:23.010361 2627 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7b3fdbcf97b849e591a68eac36be89bc [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:59:23.010546 2627 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7b3fdbcf97b849e591a68eac36be89bc [term 1 LEADER]: Becoming Leader. State: Replica: 7b3fdbcf97b849e591a68eac36be89bc, State: Running, Role: LEADER
I20260811 18:59:23.010869 2627 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7b3fdbcf97b849e591a68eac36be89bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b3fdbcf97b849e591a68eac36be89bc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36027 } }
I20260811 18:59:23.011354 2612 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7b3fdbcf97b849e591a68eac36be89bc [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:59:23.012321 2629 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b3fdbcf97b849e591a68eac36be89bc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7b3fdbcf97b849e591a68eac36be89bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b3fdbcf97b849e591a68eac36be89bc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36027 } } }
I20260811 18:59:23.012444 2629 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b3fdbcf97b849e591a68eac36be89bc [sys.catalog]: This master's current role is: LEADER
I20260811 18:59:23.012738 2631 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b3fdbcf97b849e591a68eac36be89bc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7b3fdbcf97b849e591a68eac36be89bc. Latest consensus state: current_term: 1 leader_uuid: "7b3fdbcf97b849e591a68eac36be89bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b3fdbcf97b849e591a68eac36be89bc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36027 } } }
I20260811 18:59:23.012849 2631 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b3fdbcf97b849e591a68eac36be89bc [sys.catalog]: This master's current role is: LEADER
I20260811 18:59:23.012884 2641 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:59:23.015931 2641 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:59:23.021416 2641 catalog_manager.cc:1347] Generated new cluster ID: b31bc195a311419eb50e9aabba6bb1a8
I20260811 18:59:23.021473 2641 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:59:23.028429 2641 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:59:23.028713 2641 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:59:23.038326 2641 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7b3fdbcf97b849e591a68eac36be89bc: Generated new TSK 0
I20260811 18:59:23.039039 2641 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:59:23.044061 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 2482
2026-08-11T18:59:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1523 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474496157106-28370-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:59:23 dist-test-slave-2j7r krb5kdc[2670](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:59:23 dist-test-slave-2j7r krb5kdc[2670](info): set up 2 sockets
Aug 11 18:59:23 dist-test-slave-2j7r krb5kdc[2670](info): commencing operation
krb5kdc: starting...
W20260811 18:59:24.251016 28370 mini_kdc.cc:121] Time spent starting KDC: real 1.178s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474496157106-28370-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474496157106-28370-0/krb5kdc/test-user.keytab.
Aug 11 18:59:24 dist-test-slave-2j7r krb5kdc[2670](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474764, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:59:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:59:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474496157106-28370-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474496157106-28370-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 18:59:24.373463 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474496157106-28370-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474496157106-28370-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474496157106-28370-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45255
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474496157106-28370-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45255
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:59:24.481907 2691 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:59:24.482179 2691 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:59:24.482239 2691 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:59:24.486315 2691 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:59:24.486380 2691 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:59:24.486407 2691 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:59:24.486428 2691 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:59:24.486457 2691 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:59:24.491729 2691 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474496157106-28370-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474496157106-28370-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45255
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474496157106-28370-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45255
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2691
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474496157106-28370-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:59:24.492760 2691 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:59:24.493778 2691 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:59:24.501868 2705 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:24.501876 2714 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:24.501959 2708 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:59:24.502164 2691 server_base.cc:1034] running on GCE node
I20260811 18:59:24.502696 2691 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:59:24.503747 2691 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:59:24.504901 2691 hybrid_clock.cc:648] HybridClock initialized: now 1786474764504883 us; error 40 us; skew 500 ppm
Aug 11 18:59:24 dist-test-slave-2j7r krb5kdc[2670](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474764, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:59:24.507606 2691 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 18:59:24.508494 2691 webserver.cc:460] Webserver started at http://127.0.0.1:45427/ using document root <none> and password file <none>
I20260811 18:59:24.509021 2691 fs_manager.cc:359] Metadata directory not provided
I20260811 18:59:24.509109 2691 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:59:24.509333 2691 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:59:24.511348 2691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474496157106-28370-0/minicluster-data/master-0/data/instance:
uuid: "4b46dc893b8e436b8c6d6269c3877874"
format_stamp: "Formatted at 2026-08-11 18:59:24 on dist-test-slave-2j7r"
server_key: "2d5426fe0e57facf5c79b612255362f3"
server_key_iv: "0af09e9143cea0bbd94ff6137af32a0c"
server_key_version: "encryptionkey@0"
I20260811 18:59:24.511891 2691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474496157106-28370-0/minicluster-data/master-0/wal/instance:
uuid: "4b46dc893b8e436b8c6d6269c3877874"
format_stamp: "Formatted at 2026-08-11 18:59:24 on dist-test-slave-2j7r"
server_key: "2d5426fe0e57facf5c79b612255362f3"
server_key_iv: "0af09e9143cea0bbd94ff6137af32a0c"
server_key_version: "encryptionkey@0"
I20260811 18:59:24.515508 2691 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260811 18:59:24.517802 2733 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:24.518980 2691 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 18:59:24.519094 2691 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474496157106-28370-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474496157106-28370-0/minicluster-data/master-0/wal
uuid: "4b46dc893b8e436b8c6d6269c3877874"
format_stamp: "Formatted at 2026-08-11 18:59:24 on dist-test-slave-2j7r"
server_key: "2d5426fe0e57facf5c79b612255362f3"
server_key_iv: "0af09e9143cea0bbd94ff6137af32a0c"
server_key_version: "encryptionkey@0"
I20260811 18:59:24.519184 2691 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474496157106-28370-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474496157106-28370-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474496157106-28370-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:59:24.534482 2691 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:59:24.537388 2691 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:59:24.537568 2691 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:59:24.544157 2691 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45255
I20260811 18:59:24.544166 2841 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45255 every 8 connection(s)
I20260811 18:59:24.545064 2691 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
I20260811 18:59:24.547751 2842 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:59:24.548836 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 2691
I20260811 18:59:24.548979 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786474496157106-28370-0/minicluster-data/master-0/wal/instance
I20260811 18:59:24.549286 28370 external_mini_cluster.cc:1428] Setting key 077e0cd4247dd0e576539c380f7948d9
I20260811 18:59:24.554111 2842 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b46dc893b8e436b8c6d6269c3877874: Bootstrap starting.
I20260811 18:59:24.556596 2842 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4b46dc893b8e436b8c6d6269c3877874: Neither blocks nor log segments found. Creating new log.
I20260811 18:59:24.557370 2842 log.cc:824] T 00000000000000000000000000000000 P 4b46dc893b8e436b8c6d6269c3877874: Log is configured to *not* fsync() on all Append() calls
Aug 11 18:59:24 dist-test-slave-2j7r krb5kdc[2670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474764, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 18:59:24.559530 2842 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b46dc893b8e436b8c6d6269c3877874: No bootstrap required, opened a new log
I20260811 18:59:24.563208 2842 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4b46dc893b8e436b8c6d6269c3877874 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b46dc893b8e436b8c6d6269c3877874" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45255 } }
I20260811 18:59:24.563439 2842 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4b46dc893b8e436b8c6d6269c3877874 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:59:24.563498 2842 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4b46dc893b8e436b8c6d6269c3877874 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b46dc893b8e436b8c6d6269c3877874, State: Initialized, Role: FOLLOWER
I20260811 18:59:24.563766 2846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:24.550720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.230:41593 (local address 127.0.0.1:45255)
0811 18:59:24.551130 (+ 410us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:24.551148 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:24.551189 (+ 41us) server_negotiation.cc:407] Connection header received
0811 18:59:24.551984 (+ 795us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:24.552007 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:24.552428 (+ 421us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:24.552775 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:24.553593 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:24.554987 (+ 1394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:24.555827 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:24.556494 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:24.556653 (+ 159us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:24.559320 (+ 2667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:24.559325 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:24.559337 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:59:24.559363 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:24.560871 (+ 1508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:24.561464 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:24.561469 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:24.561472 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:24.561582 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:24.561901 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:24.561903 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:24.561904 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:24.562264 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:24.562432 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:24.562728 (+ 296us) server_negotiation.cc:299] Negotiation successful
0811 18:59:24.562967 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":268,"thread_start_us":108,"threads_started":1}
I20260811 18:59:24.563913 2842 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4b46dc893b8e436b8c6d6269c3877874 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b46dc893b8e436b8c6d6269c3877874" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45255 } }
I20260811 18:59:24.564051 2842 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4b46dc893b8e436b8c6d6269c3877874 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:59:24.564112 2842 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4b46dc893b8e436b8c6d6269c3877874 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:59:24.564205 2842 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4b46dc893b8e436b8c6d6269c3877874 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:59:24.565155 2842 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4b46dc893b8e436b8c6d6269c3877874 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b46dc893b8e436b8c6d6269c3877874" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45255 } }
I20260811 18:59:24.565506 2842 leader_election.cc:302] T 00000000000000000000000000000000 P 4b46dc893b8e436b8c6d6269c3877874 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4b46dc893b8e436b8c6d6269c3877874; no voters:
I20260811 18:59:24.565833 2842 leader_election.cc:288] T 00000000000000000000000000000000 P 4b46dc893b8e436b8c6d6269c3877874 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:59:24.565912 2849 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4b46dc893b8e436b8c6d6269c3877874 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:59:24.566138 2849 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4b46dc893b8e436b8c6d6269c3877874 [term 1 LEADER]: Becoming Leader. State: Replica: 4b46dc893b8e436b8c6d6269c3877874, State: Running, Role: LEADER
I20260811 18:59:24.566471 2849 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4b46dc893b8e436b8c6d6269c3877874 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b46dc893b8e436b8c6d6269c3877874" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45255 } }
I20260811 18:59:24.566776 28370 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:59:24.567054 2842 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4b46dc893b8e436b8c6d6269c3877874 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:59:24.567580 2849 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b46dc893b8e436b8c6d6269c3877874 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4b46dc893b8e436b8c6d6269c3877874. Latest consensus state: current_term: 1 leader_uuid: "4b46dc893b8e436b8c6d6269c3877874" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b46dc893b8e436b8c6d6269c3877874" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45255 } } }
I20260811 18:59:24.567749 2849 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b46dc893b8e436b8c6d6269c3877874 [sys.catalog]: This master's current role is: LEADER
I20260811 18:59:24.568159 2869 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:59:24.569130 2857 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b46dc893b8e436b8c6d6269c3877874 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4b46dc893b8e436b8c6d6269c3877874" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b46dc893b8e436b8c6d6269c3877874" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45255 } } }
I20260811 18:59:24.569243 2857 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b46dc893b8e436b8c6d6269c3877874 [sys.catalog]: This master's current role is: LEADER
I20260811 18:59:24.572047 2869 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:59:24.577811 2869 catalog_manager.cc:1347] Generated new cluster ID: c869a236f90544c8a885b51dd96fe0eb
I20260811 18:59:24.577873 2869 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:59:24.578294 2846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:24.569471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.230:55465 (local address 127.0.0.1:45255)
0811 18:59:24.569566 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:24.569570 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:24.569696 (+ 126us) server_negotiation.cc:407] Connection header received
0811 18:59:24.569836 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:24.569839 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:24.569886 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:24.569969 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:24.570682 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:24.571405 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:24.572519 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:24.572889 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:24.572969 (+ 80us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:24.574742 (+ 1773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:24.574746 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:24.574749 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:59:24.574775 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:24.576381 (+ 1606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:24.576920 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:24.576923 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:24.576926 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:24.577010 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:24.577384 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:24.577388 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:24.577390 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:24.577603 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:24.577749 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:24.578050 (+ 301us) server_negotiation.cc:299] Negotiation successful
0811 18:59:24.578150 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260811 18:59:24.590507 2869 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:59:24.591384 2869 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:59:24.597035 2869 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4b46dc893b8e436b8c6d6269c3877874: Generated new TSK 0
I20260811 18:59:24.597689 2869 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:59:24.607697 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 2691
W20260811 18:59:24.625064 2667 connection.cc:535] client connection to 127.0.0.1:45255 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45255): Connection reset by peer (error 104)
2026-08-11T18:59:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1566 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474496157106-28370-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:59:24 dist-test-slave-2j7r krb5kdc[2906](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:59:24 dist-test-slave-2j7r krb5kdc[2906](info): set up 2 sockets
Aug 11 18:59:24 dist-test-slave-2j7r krb5kdc[2906](info): commencing operation
krb5kdc: starting...
W20260811 18:59:25.816404 28370 mini_kdc.cc:121] Time spent starting KDC: real 1.177s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474496157106-28370-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474496157106-28370-0/krb5kdc/test-user.keytab.
Aug 11 18:59:25 dist-test-slave-2j7r krb5kdc[2906](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474765, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:59:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:59:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474496157106-28370-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474496157106-28370-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 18:59:25.940014 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474496157106-28370-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474496157106-28370-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474496157106-28370-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43329
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474496157106-28370-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43329
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:59:26.056998 2934 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:59:26.057289 2934 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:59:26.057339 2934 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:59:26.061539 2934 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:59:26.061594 2934 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:59:26.061612 2934 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:59:26.061632 2934 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:59:26.061650 2934 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:59:26.066880 2934 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474496157106-28370-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474496157106-28370-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43329
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474496157106-28370-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43329
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2934
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474496157106-28370-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:59:26.067862 2934 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:59:26.068811 2934 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:59:26.076896 2946 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:26.076922 2948 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:26.076910 2944 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:59:26.077068 2934 server_base.cc:1034] running on GCE node
I20260811 18:59:26.077720 2934 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:59:26.078763 2934 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:59:26.079909 2934 hybrid_clock.cc:648] HybridClock initialized: now 1786474766079888 us; error 35 us; skew 500 ppm
Aug 11 18:59:26 dist-test-slave-2j7r krb5kdc[2906](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474766, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:59:26.082566 2934 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 18:59:26.083379 2934 webserver.cc:460] Webserver started at http://127.0.0.1:43691/ using document root <none> and password file <none>
I20260811 18:59:26.083895 2934 fs_manager.cc:359] Metadata directory not provided
I20260811 18:59:26.083961 2934 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:59:26.084146 2934 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:59:26.086165 2934 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474496157106-28370-0/minicluster-data/master-0/data/instance:
uuid: "e147fd9466f64104b81d68258e84f39c"
format_stamp: "Formatted at 2026-08-11 18:59:26 on dist-test-slave-2j7r"
server_key: "7159a73b9ca689bc0f55533a8593a101"
server_key_iv: "06d4db50c2a7b0697979c1c8170f5782"
server_key_version: "encryptionkey@0"
I20260811 18:59:26.086725 2934 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474496157106-28370-0/minicluster-data/master-0/wal/instance:
uuid: "e147fd9466f64104b81d68258e84f39c"
format_stamp: "Formatted at 2026-08-11 18:59:26 on dist-test-slave-2j7r"
server_key: "7159a73b9ca689bc0f55533a8593a101"
server_key_iv: "06d4db50c2a7b0697979c1c8170f5782"
server_key_version: "encryptionkey@0"
I20260811 18:59:26.090217 2934 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 18:59:26.092463 2957 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:26.093640 2934 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 18:59:26.093751 2934 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474496157106-28370-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474496157106-28370-0/minicluster-data/master-0/wal
uuid: "e147fd9466f64104b81d68258e84f39c"
format_stamp: "Formatted at 2026-08-11 18:59:26 on dist-test-slave-2j7r"
server_key: "7159a73b9ca689bc0f55533a8593a101"
server_key_iv: "06d4db50c2a7b0697979c1c8170f5782"
server_key_version: "encryptionkey@0"
I20260811 18:59:26.093840 2934 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474496157106-28370-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474496157106-28370-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474496157106-28370-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:59:26.113296 2934 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:59:26.116147 2934 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:59:26.116312 2934 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:59:26.122932 2934 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43329
I20260811 18:59:26.122979 3061 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43329 every 8 connection(s)
I20260811 18:59:26.123865 2934 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
I20260811 18:59:26.126648 3062 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:59:26.126915 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 2934
I20260811 18:59:26.127053 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786474496157106-28370-0/minicluster-data/master-0/wal/instance
I20260811 18:59:26.127314 28370 external_mini_cluster.cc:1428] Setting key 5b738d11b68ca396257f7910afb98b2b
I20260811 18:59:26.133826 3062 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e147fd9466f64104b81d68258e84f39c: Bootstrap starting.
I20260811 18:59:26.136157 3062 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e147fd9466f64104b81d68258e84f39c: Neither blocks nor log segments found. Creating new log.
I20260811 18:59:26.136678 3067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:26.128671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.230:45225 (local address 127.0.0.1:43329)
0811 18:59:26.129115 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:26.129134 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:26.129180 (+ 46us) server_negotiation.cc:407] Connection header received
0811 18:59:26.129906 (+ 726us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:26.129930 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:26.130366 (+ 436us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:26.130742 (+ 376us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:26.131127 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:26.132585 (+ 1458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:26.133498 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:26.134004 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:26.134154 (+ 150us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:26.135059 (+ 905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:26.135064 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:26.135075 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:59:26.135437 (+ 362us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:26.135489 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:59:26.135510 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:26.135627 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:26.135805 (+ 178us) server_negotiation.cc:299] Negotiation successful
0811 18:59:26.135951 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273,"thread_start_us":111,"threads_started":1}
I20260811 18:59:26.136919 3062 log.cc:824] T 00000000000000000000000000000000 P e147fd9466f64104b81d68258e84f39c: Log is configured to *not* fsync() on all Append() calls
I20260811 18:59:26.138875 3062 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e147fd9466f64104b81d68258e84f39c: No bootstrap required, opened a new log
I20260811 18:59:26.139488 28370 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:59:26.142627 3062 raft_consensus.cc:348] T 00000000000000000000000000000000 P e147fd9466f64104b81d68258e84f39c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e147fd9466f64104b81d68258e84f39c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43329 } }
I20260811 18:59:26.142885 3062 raft_consensus.cc:374] T 00000000000000000000000000000000 P e147fd9466f64104b81d68258e84f39c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:59:26.142948 3062 raft_consensus.cc:729] T 00000000000000000000000000000000 P e147fd9466f64104b81d68258e84f39c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e147fd9466f64104b81d68258e84f39c, State: Initialized, Role: FOLLOWER
I20260811 18:59:26.143399 3062 consensus_queue.cc:260] T 00000000000000000000000000000000 P e147fd9466f64104b81d68258e84f39c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e147fd9466f64104b81d68258e84f39c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43329 } }
I20260811 18:59:26.143553 3062 raft_consensus.cc:388] T 00000000000000000000000000000000 P e147fd9466f64104b81d68258e84f39c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:59:26.143618 3062 raft_consensus.cc:482] T 00000000000000000000000000000000 P e147fd9466f64104b81d68258e84f39c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:59:26.143723 3062 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e147fd9466f64104b81d68258e84f39c [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:59:26.144691 3062 raft_consensus.cc:504] T 00000000000000000000000000000000 P e147fd9466f64104b81d68258e84f39c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e147fd9466f64104b81d68258e84f39c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43329 } }
I20260811 18:59:26.145009 3062 leader_election.cc:302] T 00000000000000000000000000000000 P e147fd9466f64104b81d68258e84f39c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e147fd9466f64104b81d68258e84f39c; no voters:
I20260811 18:59:26.145325 3062 leader_election.cc:288] T 00000000000000000000000000000000 P e147fd9466f64104b81d68258e84f39c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:59:26.145421 3078 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e147fd9466f64104b81d68258e84f39c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:59:26.145434 3067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:26.141418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.230:32779 (local address 127.0.0.1:43329)
0811 18:59:26.141540 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:26.141544 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:26.141673 (+ 129us) server_negotiation.cc:407] Connection header received
0811 18:59:26.141785 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:26.141787 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:26.141823 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:26.141902 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:26.142169 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:26.142665 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:26.143764 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:26.144105 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:26.144203 (+ 98us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:26.144920 (+ 717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:26.144924 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:26.144927 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:59:26.144940 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:26.144966 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:59:26.144976 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:26.145044 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:26.145219 (+ 175us) server_negotiation.cc:299] Negotiation successful
0811 18:59:26.145251 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 18:59:26.145640 3078 raft_consensus.cc:686] T 00000000000000000000000000000000 P e147fd9466f64104b81d68258e84f39c [term 1 LEADER]: Becoming Leader. State: Replica: e147fd9466f64104b81d68258e84f39c, State: Running, Role: LEADER
I20260811 18:59:26.145962 3078 consensus_queue.cc:237] T 00000000000000000000000000000000 P e147fd9466f64104b81d68258e84f39c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e147fd9466f64104b81d68258e84f39c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43329 } }
I20260811 18:59:26.146669 3062 sys_catalog.cc:564] T 00000000000000000000000000000000 P e147fd9466f64104b81d68258e84f39c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:59:26.147751 3091 sys_catalog.cc:455] T 00000000000000000000000000000000 P e147fd9466f64104b81d68258e84f39c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e147fd9466f64104b81d68258e84f39c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e147fd9466f64104b81d68258e84f39c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43329 } } }
I20260811 18:59:26.147725 3087 sys_catalog.cc:455] T 00000000000000000000000000000000 P e147fd9466f64104b81d68258e84f39c [sys.catalog]: SysCatalogTable state changed. Reason: New leader e147fd9466f64104b81d68258e84f39c. Latest consensus state: current_term: 1 leader_uuid: "e147fd9466f64104b81d68258e84f39c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e147fd9466f64104b81d68258e84f39c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43329 } } }
I20260811 18:59:26.147858 3087 sys_catalog.cc:458] T 00000000000000000000000000000000 P e147fd9466f64104b81d68258e84f39c [sys.catalog]: This master's current role is: LEADER
I20260811 18:59:26.147858 3091 sys_catalog.cc:458] T 00000000000000000000000000000000 P e147fd9466f64104b81d68258e84f39c [sys.catalog]: This master's current role is: LEADER
I20260811 18:59:26.148269 3109 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:59:26.151075 3109 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:59:26.156844 3109 catalog_manager.cc:1347] Generated new cluster ID: 5db2d4e23c6c4c6880afa91a9ef43d69
I20260811 18:59:26.156901 3109 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:59:26.176532 3109 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:59:26.177699 3109 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:59:26.189260 3109 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e147fd9466f64104b81d68258e84f39c: Generated new TSK 0
I20260811 18:59:26.189858 3109 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:59:26.201913 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 2934
W20260811 18:59:26.213586 2904 connection.cc:535] client connection to 127.0.0.1:43329 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43329): Connection reset by peer (error 104)
2026-08-11T18:59:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1588 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474496157106-28370-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:59:26 dist-test-slave-2j7r krb5kdc[3139](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:59:26 dist-test-slave-2j7r krb5kdc[3139](info): set up 2 sockets
Aug 11 18:59:26 dist-test-slave-2j7r krb5kdc[3139](info): commencing operation
krb5kdc: starting...
W20260811 18:59:27.405452 28370 mini_kdc.cc:121] Time spent starting KDC: real 1.178s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474496157106-28370-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474496157106-28370-0/krb5kdc/test-user.keytab.
Aug 11 18:59:27 dist-test-slave-2j7r krb5kdc[3139](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474767, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:59:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:59:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474496157106-28370-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474496157106-28370-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 18:59:27.525585 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474496157106-28370-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474496157106-28370-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474496157106-28370-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44295
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474496157106-28370-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44295
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:59:27.635334 3170 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:59:27.635571 3170 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:59:27.635625 3170 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:59:27.640030 3170 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:59:27.640098 3170 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:59:27.640129 3170 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:59:27.640151 3170 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:59:27.640172 3170 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:59:27.645355 3170 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474496157106-28370-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474496157106-28370-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44295
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474496157106-28370-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44295
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3170
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474496157106-28370-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:59:27.646337 3170 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:59:27.647274 3170 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:59:27.655658 3179 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:27.655633 3183 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:27.655642 3180 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:59:27.655930 3170 server_base.cc:1034] running on GCE node
I20260811 18:59:27.656453 3170 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:59:27.657536 3170 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:59:27.658695 3170 hybrid_clock.cc:648] HybridClock initialized: now 1786474767658681 us; error 25 us; skew 500 ppm
Aug 11 18:59:27 dist-test-slave-2j7r krb5kdc[3139](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474767, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:59:27.661509 3170 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 18:59:27.662314 3170 webserver.cc:460] Webserver started at http://127.0.0.1:44839/ using document root <none> and password file <none>
I20260811 18:59:27.662806 3170 fs_manager.cc:359] Metadata directory not provided
I20260811 18:59:27.662860 3170 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:59:27.663028 3170 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:59:27.664990 3170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474496157106-28370-0/minicluster-data/master-0/data/instance:
uuid: "32ec152a918d4df38da6bc66c781df2a"
format_stamp: "Formatted at 2026-08-11 18:59:27 on dist-test-slave-2j7r"
server_key: "9cd2767c8ca4b5060c230a103c7bc7bd"
server_key_iv: "b97f7228199e8f583f5bc3e659b4dad4"
server_key_version: "encryptionkey@0"
I20260811 18:59:27.665552 3170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474496157106-28370-0/minicluster-data/master-0/wal/instance:
uuid: "32ec152a918d4df38da6bc66c781df2a"
format_stamp: "Formatted at 2026-08-11 18:59:27 on dist-test-slave-2j7r"
server_key: "9cd2767c8ca4b5060c230a103c7bc7bd"
server_key_iv: "b97f7228199e8f583f5bc3e659b4dad4"
server_key_version: "encryptionkey@0"
I20260811 18:59:27.669055 3170 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:59:27.671424 3193 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:27.672616 3170 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 18:59:27.672732 3170 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474496157106-28370-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474496157106-28370-0/minicluster-data/master-0/wal
uuid: "32ec152a918d4df38da6bc66c781df2a"
format_stamp: "Formatted at 2026-08-11 18:59:27 on dist-test-slave-2j7r"
server_key: "9cd2767c8ca4b5060c230a103c7bc7bd"
server_key_iv: "b97f7228199e8f583f5bc3e659b4dad4"
server_key_version: "encryptionkey@0"
I20260811 18:59:27.672821 3170 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474496157106-28370-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474496157106-28370-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474496157106-28370-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:59:27.677201 3170 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:59:27.680135 3170 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:59:27.680297 3170 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:59:27.687314 3170 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44295
I20260811 18:59:27.687345 3314 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44295 every 8 connection(s)
I20260811 18:59:27.688194 3170 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
I20260811 18:59:27.690996 3315 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:59:27.691985 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 3170
I20260811 18:59:27.692111 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786474496157106-28370-0/minicluster-data/master-0/wal/instance
I20260811 18:59:27.692394 28370 external_mini_cluster.cc:1428] Setting key b6f85c56a68e9f2c2609203a1651ed97
I20260811 18:59:27.697381 3315 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32ec152a918d4df38da6bc66c781df2a: Bootstrap starting.
I20260811 18:59:27.697487 3319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:27.693757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.230:43913 (local address 127.0.0.1:44295)
0811 18:59:27.694221 (+ 464us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:27.694239 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:27.694283 (+ 44us) server_negotiation.cc:407] Connection header received
0811 18:59:27.694916 (+ 633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:27.694933 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:27.695207 (+ 274us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:27.695510 (+ 303us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:27.696280 (+ 770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:27.696285 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:27.696297 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:59:27.696308 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:27.696362 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:59:27.696381 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:27.696509 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:27.696588 (+ 79us) server_negotiation.cc:299] Negotiation successful
0811 18:59:27.696739 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":143,"threads_started":1}
I20260811 18:59:27.699957 3315 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 32ec152a918d4df38da6bc66c781df2a: Neither blocks nor log segments found. Creating new log.
I20260811 18:59:27.700320 28370 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:59:27.700781 3315 log.cc:824] T 00000000000000000000000000000000 P 32ec152a918d4df38da6bc66c781df2a: Log is configured to *not* fsync() on all Append() calls
I20260811 18:59:27.702981 3315 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32ec152a918d4df38da6bc66c781df2a: No bootstrap required, opened a new log
I20260811 18:59:27.703891 3319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:27.702336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.230:35281 (local address 127.0.0.1:44295)
0811 18:59:27.702439 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:27.702442 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:27.702670 (+ 228us) server_negotiation.cc:407] Connection header received
0811 18:59:27.702814 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:27.702816 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:27.702844 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:27.702913 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:27.703548 (+ 635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:27.703551 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:27.703553 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:59:27.703564 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:27.703588 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:59:27.703597 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:27.703665 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:27.703759 (+ 94us) server_negotiation.cc:299] Negotiation successful
0811 18:59:27.703788 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260811 18:59:27.706907 3315 raft_consensus.cc:348] T 00000000000000000000000000000000 P 32ec152a918d4df38da6bc66c781df2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32ec152a918d4df38da6bc66c781df2a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44295 } }
I20260811 18:59:27.707120 3315 raft_consensus.cc:374] T 00000000000000000000000000000000 P 32ec152a918d4df38da6bc66c781df2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:59:27.707159 3315 raft_consensus.cc:729] T 00000000000000000000000000000000 P 32ec152a918d4df38da6bc66c781df2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32ec152a918d4df38da6bc66c781df2a, State: Initialized, Role: FOLLOWER
I20260811 18:59:27.707510 3315 consensus_queue.cc:260] T 00000000000000000000000000000000 P 32ec152a918d4df38da6bc66c781df2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32ec152a918d4df38da6bc66c781df2a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44295 } }
I20260811 18:59:27.707623 3315 raft_consensus.cc:388] T 00000000000000000000000000000000 P 32ec152a918d4df38da6bc66c781df2a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:59:27.707667 3315 raft_consensus.cc:482] T 00000000000000000000000000000000 P 32ec152a918d4df38da6bc66c781df2a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:59:27.707736 3315 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 32ec152a918d4df38da6bc66c781df2a [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:59:27.708595 3315 raft_consensus.cc:504] T 00000000000000000000000000000000 P 32ec152a918d4df38da6bc66c781df2a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32ec152a918d4df38da6bc66c781df2a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44295 } }
I20260811 18:59:27.708915 3315 leader_election.cc:302] T 00000000000000000000000000000000 P 32ec152a918d4df38da6bc66c781df2a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 32ec152a918d4df38da6bc66c781df2a; no voters:
I20260811 18:59:27.709230 3315 leader_election.cc:288] T 00000000000000000000000000000000 P 32ec152a918d4df38da6bc66c781df2a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:59:27.709333 3331 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 32ec152a918d4df38da6bc66c781df2a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:59:27.709540 3331 raft_consensus.cc:686] T 00000000000000000000000000000000 P 32ec152a918d4df38da6bc66c781df2a [term 1 LEADER]: Becoming Leader. State: Replica: 32ec152a918d4df38da6bc66c781df2a, State: Running, Role: LEADER
I20260811 18:59:27.709898 3331 consensus_queue.cc:237] T 00000000000000000000000000000000 P 32ec152a918d4df38da6bc66c781df2a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32ec152a918d4df38da6bc66c781df2a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44295 } }
I20260811 18:59:27.710374 3315 sys_catalog.cc:564] T 00000000000000000000000000000000 P 32ec152a918d4df38da6bc66c781df2a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:59:27.711392 3334 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32ec152a918d4df38da6bc66c781df2a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 32ec152a918d4df38da6bc66c781df2a. Latest consensus state: current_term: 1 leader_uuid: "32ec152a918d4df38da6bc66c781df2a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32ec152a918d4df38da6bc66c781df2a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44295 } } }
I20260811 18:59:27.711519 3334 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32ec152a918d4df38da6bc66c781df2a [sys.catalog]: This master's current role is: LEADER
I20260811 18:59:27.711915 3346 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:59:27.711860 3332 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32ec152a918d4df38da6bc66c781df2a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "32ec152a918d4df38da6bc66c781df2a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32ec152a918d4df38da6bc66c781df2a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44295 } } }
I20260811 18:59:27.712538 3332 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32ec152a918d4df38da6bc66c781df2a [sys.catalog]: This master's current role is: LEADER
I20260811 18:59:27.715106 3346 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:59:27.721372 3346 catalog_manager.cc:1347] Generated new cluster ID: eeb1ebcea0bf42bb860baaee41472aec
I20260811 18:59:27.721468 3346 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:59:27.732543 3346 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:59:27.732822 3346 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:59:27.740911 3346 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 32ec152a918d4df38da6bc66c781df2a: Generated new TSK 0
I20260811 18:59:27.741530 3346 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:59:27.748204 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 3170
2026-08-11T18:59:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1553 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474496157106-28370-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:59:27 dist-test-slave-2j7r krb5kdc[3368](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:59:27 dist-test-slave-2j7r krb5kdc[3368](info): set up 2 sockets
Aug 11 18:59:27 dist-test-slave-2j7r krb5kdc[3368](info): commencing operation
krb5kdc: starting...
W20260811 18:59:28.949246 28370 mini_kdc.cc:121] Time spent starting KDC: real 1.168s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474496157106-28370-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474496157106-28370-0/krb5kdc/test-user.keytab.
Aug 11 18:59:29 dist-test-slave-2j7r krb5kdc[3368](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474769, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:59:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:59:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474496157106-28370-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474496157106-28370-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 18:59:29.071483 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474496157106-28370-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474496157106-28370-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474496157106-28370-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36013
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474496157106-28370-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36013
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:59:29.179415 3391 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:59:29.179657 3391 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:59:29.179695 3391 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:59:29.183647 3391 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:59:29.183701 3391 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:59:29.183718 3391 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:59:29.183733 3391 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:59:29.183746 3391 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:59:29.188748 3391 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474496157106-28370-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474496157106-28370-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36013
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474496157106-28370-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36013
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3391
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474496157106-28370-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:59:29.189752 3391 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:59:29.190670 3391 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:59:29.198796 3417 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:29.198830 3402 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:29.198987 3407 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:59:29.199009 3391 server_base.cc:1034] running on GCE node
I20260811 18:59:29.199680 3391 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:59:29.200734 3391 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:59:29.201897 3391 hybrid_clock.cc:648] HybridClock initialized: now 1786474769201882 us; error 32 us; skew 500 ppm
Aug 11 18:59:29 dist-test-slave-2j7r krb5kdc[3368](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474769, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:59:29.207432 3391 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 18:59:29.208331 3391 webserver.cc:460] Webserver started at http://127.0.0.1:42157/ using document root <none> and password file <none>
I20260811 18:59:29.208882 3391 fs_manager.cc:359] Metadata directory not provided
I20260811 18:59:29.208949 3391 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:59:29.209199 3391 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:59:29.211229 3391 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474496157106-28370-0/minicluster-data/master-0/data/instance:
uuid: "69d808cf4ad14ea9b9901fc79f6c7fa9"
format_stamp: "Formatted at 2026-08-11 18:59:29 on dist-test-slave-2j7r"
server_key: "490adb4773a200b7975f1c4dd39d7074"
server_key_iv: "fe2ed98b8696ea7f7597c2743f8535e4"
server_key_version: "encryptionkey@0"
I20260811 18:59:29.211815 3391 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474496157106-28370-0/minicluster-data/master-0/wal/instance:
uuid: "69d808cf4ad14ea9b9901fc79f6c7fa9"
format_stamp: "Formatted at 2026-08-11 18:59:29 on dist-test-slave-2j7r"
server_key: "490adb4773a200b7975f1c4dd39d7074"
server_key_iv: "fe2ed98b8696ea7f7597c2743f8535e4"
server_key_version: "encryptionkey@0"
I20260811 18:59:29.215579 3391 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 18:59:29.218021 3430 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:29.219204 3391 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 18:59:29.219321 3391 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474496157106-28370-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474496157106-28370-0/minicluster-data/master-0/wal
uuid: "69d808cf4ad14ea9b9901fc79f6c7fa9"
format_stamp: "Formatted at 2026-08-11 18:59:29 on dist-test-slave-2j7r"
server_key: "490adb4773a200b7975f1c4dd39d7074"
server_key_iv: "fe2ed98b8696ea7f7597c2743f8535e4"
server_key_version: "encryptionkey@0"
I20260811 18:59:29.219422 3391 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474496157106-28370-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474496157106-28370-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474496157106-28370-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:59:29.247133 3391 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:59:29.250174 3391 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:59:29.250371 3391 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:59:29.257490 3391 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36013
I20260811 18:59:29.257524 3525 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36013 every 8 connection(s)
I20260811 18:59:29.258440 3391 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
I20260811 18:59:29.261199 3526 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:59:29.266743 3526 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69d808cf4ad14ea9b9901fc79f6c7fa9: Bootstrap starting.
I20260811 18:59:29.267694 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 3391
I20260811 18:59:29.267849 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786474496157106-28370-0/minicluster-data/master-0/wal/instance
I20260811 18:59:29.268170 28370 external_mini_cluster.cc:1428] Setting key 6320f16d59882a9dbd753667f9b75a5e
I20260811 18:59:29.269729 3526 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 69d808cf4ad14ea9b9901fc79f6c7fa9: Neither blocks nor log segments found. Creating new log.
I20260811 18:59:29.270444 3526 log.cc:824] T 00000000000000000000000000000000 P 69d808cf4ad14ea9b9901fc79f6c7fa9: Log is configured to *not* fsync() on all Append() calls
I20260811 18:59:29.272254 3526 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69d808cf4ad14ea9b9901fc79f6c7fa9: No bootstrap required, opened a new log
I20260811 18:59:29.275720 3526 raft_consensus.cc:348] T 00000000000000000000000000000000 P 69d808cf4ad14ea9b9901fc79f6c7fa9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69d808cf4ad14ea9b9901fc79f6c7fa9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36013 } }
I20260811 18:59:29.275959 3526 raft_consensus.cc:374] T 00000000000000000000000000000000 P 69d808cf4ad14ea9b9901fc79f6c7fa9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:59:29.276018 3526 raft_consensus.cc:729] T 00000000000000000000000000000000 P 69d808cf4ad14ea9b9901fc79f6c7fa9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69d808cf4ad14ea9b9901fc79f6c7fa9, State: Initialized, Role: FOLLOWER
Aug 11 18:59:29 dist-test-slave-2j7r krb5kdc[3368](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474769, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 18:59:29.276432 3526 consensus_queue.cc:260] T 00000000000000000000000000000000 P 69d808cf4ad14ea9b9901fc79f6c7fa9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69d808cf4ad14ea9b9901fc79f6c7fa9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36013 } }
I20260811 18:59:29.276580 3526 raft_consensus.cc:388] T 00000000000000000000000000000000 P 69d808cf4ad14ea9b9901fc79f6c7fa9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:59:29.276643 3526 raft_consensus.cc:482] T 00000000000000000000000000000000 P 69d808cf4ad14ea9b9901fc79f6c7fa9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:59:29.276739 3526 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 69d808cf4ad14ea9b9901fc79f6c7fa9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:59:29.277745 3526 raft_consensus.cc:504] T 00000000000000000000000000000000 P 69d808cf4ad14ea9b9901fc79f6c7fa9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69d808cf4ad14ea9b9901fc79f6c7fa9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36013 } }
I20260811 18:59:29.278064 3526 leader_election.cc:302] T 00000000000000000000000000000000 P 69d808cf4ad14ea9b9901fc79f6c7fa9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 69d808cf4ad14ea9b9901fc79f6c7fa9; no voters:
I20260811 18:59:29.278343 3526 leader_election.cc:288] T 00000000000000000000000000000000 P 69d808cf4ad14ea9b9901fc79f6c7fa9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:59:29.278471 3534 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 69d808cf4ad14ea9b9901fc79f6c7fa9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:59:29.278679 3534 raft_consensus.cc:686] T 00000000000000000000000000000000 P 69d808cf4ad14ea9b9901fc79f6c7fa9 [term 1 LEADER]: Becoming Leader. State: Replica: 69d808cf4ad14ea9b9901fc79f6c7fa9, State: Running, Role: LEADER
I20260811 18:59:29.279101 3534 consensus_queue.cc:237] T 00000000000000000000000000000000 P 69d808cf4ad14ea9b9901fc79f6c7fa9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69d808cf4ad14ea9b9901fc79f6c7fa9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36013 } }
I20260811 18:59:29.279697 3526 sys_catalog.cc:564] T 00000000000000000000000000000000 P 69d808cf4ad14ea9b9901fc79f6c7fa9 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:59:29.280838 3539 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69d808cf4ad14ea9b9901fc79f6c7fa9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 69d808cf4ad14ea9b9901fc79f6c7fa9. Latest consensus state: current_term: 1 leader_uuid: "69d808cf4ad14ea9b9901fc79f6c7fa9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69d808cf4ad14ea9b9901fc79f6c7fa9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36013 } } }
I20260811 18:59:29.280966 3539 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69d808cf4ad14ea9b9901fc79f6c7fa9 [sys.catalog]: This master's current role is: LEADER
I20260811 18:59:29.281167 3536 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69d808cf4ad14ea9b9901fc79f6c7fa9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "69d808cf4ad14ea9b9901fc79f6c7fa9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69d808cf4ad14ea9b9901fc79f6c7fa9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36013 } } }
I20260811 18:59:29.281510 3536 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69d808cf4ad14ea9b9901fc79f6c7fa9 [sys.catalog]: This master's current role is: LEADER
I20260811 18:59:29.283176 3531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:29.269311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.230:34311 (local address 127.0.0.1:36013)
0811 18:59:29.269747 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:29.269763 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:29.269800 (+ 37us) server_negotiation.cc:407] Connection header received
0811 18:59:29.270471 (+ 671us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:29.270488 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:29.270852 (+ 364us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:29.271160 (+ 308us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:29.271970 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:29.272964 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:29.273831 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:29.274305 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:29.274464 (+ 159us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:29.277103 (+ 2639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:29.277111 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:29.277128 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:59:29.277160 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:29.279046 (+ 1886us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:29.279696 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:29.279702 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:29.279708 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:29.279856 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:29.280259 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:29.280263 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:29.280265 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:29.280791 (+ 526us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:29.280997 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:29.281922 (+ 925us) server_negotiation.cc:299] Negotiation successful
0811 18:59:29.282159 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":287,"thread_start_us":150,"threads_started":1}
W20260811 18:59:29.285367 3566 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 69d808cf4ad14ea9b9901fc79f6c7fa9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:59:29.285457 3566 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:59:29.285534 3548 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:59:29.286429 3548 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:59:29.286458 28370 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:59:29.292166 3548 catalog_manager.cc:1347] Generated new cluster ID: 53871283902f4df3ba6ff559dcacc5f8
I20260811 18:59:29.292232 3548 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:59:29.297379 3531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:29.288567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.230:38569 (local address 127.0.0.1:36013)
0811 18:59:29.288661 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:29.288665 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:29.288933 (+ 268us) server_negotiation.cc:407] Connection header received
0811 18:59:29.289034 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:29.289038 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:29.289105 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:29.289189 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:29.289946 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:29.290773 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:29.291819 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:29.292185 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:29.292285 (+ 100us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:29.294075 (+ 1790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:29.294079 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:29.294082 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:59:29.294106 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:29.295608 (+ 1502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:29.296123 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:29.296126 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:29.296128 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:29.296200 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:29.296484 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:29.296487 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:29.296488 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:29.296662 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:29.296803 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:29.297018 (+ 215us) server_negotiation.cc:299] Negotiation successful
0811 18:59:29.297230 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260811 18:59:29.305625 3548 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:59:29.306533 3548 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:59:29.311025 3548 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 69d808cf4ad14ea9b9901fc79f6c7fa9: Generated new TSK 0
I20260811 18:59:29.311628 3548 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:59:29.319334 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 3391
W20260811 18:59:29.341899 3363 connection.cc:535] client connection to 127.0.0.1:36013 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36013): Connection reset by peer (error 104)
2026-08-11T18:59:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1574 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474496157106-28370-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:59:29 dist-test-slave-2j7r krb5kdc[3587](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:59:29 dist-test-slave-2j7r krb5kdc[3587](info): set up 2 sockets
Aug 11 18:59:29 dist-test-slave-2j7r krb5kdc[3587](info): commencing operation
krb5kdc: starting...
W20260811 18:59:30.533628 28370 mini_kdc.cc:121] Time spent starting KDC: real 1.177s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474496157106-28370-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474496157106-28370-0/krb5kdc/test-user.keytab.
Aug 11 18:59:30 dist-test-slave-2j7r krb5kdc[3587](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474770, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:59:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:59:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474496157106-28370-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474496157106-28370-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 18:59:30.651800 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474496157106-28370-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474496157106-28370-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474496157106-28370-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45273
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474496157106-28370-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45273
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:59:30.761726 3610 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:59:30.761989 3610 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:59:30.762046 3610 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:59:30.766129 3610 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:59:30.766201 3610 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:59:30.766229 3610 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:59:30.766259 3610 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:59:30.766289 3610 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:59:30.771574 3610 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474496157106-28370-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474496157106-28370-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45273
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474496157106-28370-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45273
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3610
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474496157106-28370-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:59:30.772593 3610 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:59:30.773603 3610 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:59:30.781920 3625 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:30.781917 3618 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:30.781913 3617 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:59:30.782253 3610 server_base.cc:1034] running on GCE node
I20260811 18:59:30.782855 3610 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:59:30.783979 3610 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:59:30.785168 3610 hybrid_clock.cc:648] HybridClock initialized: now 1786474770785134 us; error 44 us; skew 500 ppm
Aug 11 18:59:30 dist-test-slave-2j7r krb5kdc[3587](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474770, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:59:30.787834 3610 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 18:59:30.788674 3610 webserver.cc:460] Webserver started at http://127.0.0.1:41919/ using document root <none> and password file <none>
I20260811 18:59:30.789232 3610 fs_manager.cc:359] Metadata directory not provided
I20260811 18:59:30.789304 3610 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:59:30.789499 3610 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:59:30.791436 3610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474496157106-28370-0/minicluster-data/master-0/data/instance:
uuid: "4dc4596852c74bdc8ead835234cf75a2"
format_stamp: "Formatted at 2026-08-11 18:59:30 on dist-test-slave-2j7r"
server_key: "55fccb013ad48ed5c1df7ba07d18fc65"
server_key_iv: "48c6f45f1d8b6a37e7ba3ee888b57d2c"
server_key_version: "encryptionkey@0"
I20260811 18:59:30.791961 3610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474496157106-28370-0/minicluster-data/master-0/wal/instance:
uuid: "4dc4596852c74bdc8ead835234cf75a2"
format_stamp: "Formatted at 2026-08-11 18:59:30 on dist-test-slave-2j7r"
server_key: "55fccb013ad48ed5c1df7ba07d18fc65"
server_key_iv: "48c6f45f1d8b6a37e7ba3ee888b57d2c"
server_key_version: "encryptionkey@0"
I20260811 18:59:30.795555 3610 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 18:59:30.797837 3646 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:30.798962 3610 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 18:59:30.799077 3610 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474496157106-28370-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474496157106-28370-0/minicluster-data/master-0/wal
uuid: "4dc4596852c74bdc8ead835234cf75a2"
format_stamp: "Formatted at 2026-08-11 18:59:30 on dist-test-slave-2j7r"
server_key: "55fccb013ad48ed5c1df7ba07d18fc65"
server_key_iv: "48c6f45f1d8b6a37e7ba3ee888b57d2c"
server_key_version: "encryptionkey@0"
I20260811 18:59:30.799168 3610 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474496157106-28370-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474496157106-28370-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474496157106-28370-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:59:30.829630 3610 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:59:30.832254 3610 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:59:30.832440 3610 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:59:30.839242 3610 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45273
I20260811 18:59:30.839253 3757 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45273 every 8 connection(s)
I20260811 18:59:30.840180 3610 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
I20260811 18:59:30.842962 3760 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:59:30.847681 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 3610
I20260811 18:59:30.847828 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786474496157106-28370-0/minicluster-data/master-0/wal/instance
I20260811 18:59:30.848069 28370 external_mini_cluster.cc:1428] Setting key 7fd6e12b10fea4ffebf5518a5732d64f
I20260811 18:59:30.848526 3760 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4dc4596852c74bdc8ead835234cf75a2: Bootstrap starting.
I20260811 18:59:30.851209 3760 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4dc4596852c74bdc8ead835234cf75a2: Neither blocks nor log segments found. Creating new log.
I20260811 18:59:30.852106 3760 log.cc:824] T 00000000000000000000000000000000 P 4dc4596852c74bdc8ead835234cf75a2: Log is configured to *not* fsync() on all Append() calls
I20260811 18:59:30.854205 3760 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4dc4596852c74bdc8ead835234cf75a2: No bootstrap required, opened a new log
I20260811 18:59:30.857946 3767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:30.849459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.230:35683 (local address 127.0.0.1:45273)
0811 18:59:30.849883 (+ 424us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:30.849902 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:30.849945 (+ 43us) server_negotiation.cc:407] Connection header received
0811 18:59:30.850765 (+ 820us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:30.850787 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:30.851232 (+ 445us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:30.851617 (+ 385us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:30.852034 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:30.853394 (+ 1360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:30.854296 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:30.854961 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:30.855134 (+ 173us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:30.855862 (+ 728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:30.855868 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:30.855886 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:59:30.856428 (+ 542us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:30.856496 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:59:30.856524 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:30.856659 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:30.856783 (+ 124us) server_negotiation.cc:299] Negotiation successful
0811 18:59:30.856923 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":106,"threads_started":1}
I20260811 18:59:30.858595 3760 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4dc4596852c74bdc8ead835234cf75a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4dc4596852c74bdc8ead835234cf75a2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45273 } }
I20260811 18:59:30.858819 3760 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4dc4596852c74bdc8ead835234cf75a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:59:30.858906 3760 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4dc4596852c74bdc8ead835234cf75a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dc4596852c74bdc8ead835234cf75a2, State: Initialized, Role: FOLLOWER
I20260811 18:59:30.859354 3760 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4dc4596852c74bdc8ead835234cf75a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4dc4596852c74bdc8ead835234cf75a2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45273 } }
I20260811 18:59:30.859485 3760 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4dc4596852c74bdc8ead835234cf75a2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:59:30.859572 3760 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4dc4596852c74bdc8ead835234cf75a2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:59:30.859663 3760 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4dc4596852c74bdc8ead835234cf75a2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:59:30.860713 3760 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4dc4596852c74bdc8ead835234cf75a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4dc4596852c74bdc8ead835234cf75a2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45273 } }
I20260811 18:59:30.860788 28370 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:59:30.861066 3760 leader_election.cc:302] T 00000000000000000000000000000000 P 4dc4596852c74bdc8ead835234cf75a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4dc4596852c74bdc8ead835234cf75a2; no voters:
I20260811 18:59:30.861446 3760 leader_election.cc:288] T 00000000000000000000000000000000 P 4dc4596852c74bdc8ead835234cf75a2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:59:30.861805 3771 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4dc4596852c74bdc8ead835234cf75a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:59:30.862075 3771 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4dc4596852c74bdc8ead835234cf75a2 [term 1 LEADER]: Becoming Leader. State: Replica: 4dc4596852c74bdc8ead835234cf75a2, State: Running, Role: LEADER
I20260811 18:59:30.862480 3771 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4dc4596852c74bdc8ead835234cf75a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4dc4596852c74bdc8ead835234cf75a2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45273 } }
I20260811 18:59:30.862751 3760 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4dc4596852c74bdc8ead835234cf75a2 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:59:30.863833 3771 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4dc4596852c74bdc8ead835234cf75a2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4dc4596852c74bdc8ead835234cf75a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4dc4596852c74bdc8ead835234cf75a2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45273 } } }
I20260811 18:59:30.863956 3771 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4dc4596852c74bdc8ead835234cf75a2 [sys.catalog]: This master's current role is: LEADER
I20260811 18:59:30.864230 3771 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4dc4596852c74bdc8ead835234cf75a2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4dc4596852c74bdc8ead835234cf75a2. Latest consensus state: current_term: 1 leader_uuid: "4dc4596852c74bdc8ead835234cf75a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4dc4596852c74bdc8ead835234cf75a2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45273 } } }
I20260811 18:59:30.864331 3771 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4dc4596852c74bdc8ead835234cf75a2 [sys.catalog]: This master's current role is: LEADER
W20260811 18:59:30.867700 3810 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4dc4596852c74bdc8ead835234cf75a2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:59:30.867820 3810 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:59:30.867950 3800 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:59:30.868906 3800 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:59:30.871604 3767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:30.866056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.230:47975 (local address 127.0.0.1:45273)
0811 18:59:30.866182 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:30.866186 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:30.866926 (+ 740us) server_negotiation.cc:407] Connection header received
0811 18:59:30.867121 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:30.867124 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:30.867174 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:30.867224 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:30.867657 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:30.868439 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:30.869606 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:30.869928 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:30.870003 (+ 75us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:30.871055 (+ 1052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:30.871059 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:30.871062 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:59:30.871075 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:30.871105 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:59:30.871120 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:30.871182 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:30.871439 (+ 257us) server_negotiation.cc:299] Negotiation successful
0811 18:59:30.871476 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 18:59:30.874996 3800 catalog_manager.cc:1347] Generated new cluster ID: 3d51e12135c04eb389ff2e73b635572e
I20260811 18:59:30.875051 3800 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:59:30.882436 3800 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:59:30.883342 3800 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:59:30.895179 3800 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4dc4596852c74bdc8ead835234cf75a2: Generated new TSK 0
I20260811 18:59:30.895915 3800 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:59:30.908938 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 3610
W20260811 18:59:30.923187 3585 connection.cc:535] client connection to 127.0.0.1:45273 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45273): Connection reset by peer (error 104)
2026-08-11T18:59:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1580 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474496157106-28370-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:59:30 dist-test-slave-2j7r krb5kdc[3829](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:59:30 dist-test-slave-2j7r krb5kdc[3829](info): set up 2 sockets
Aug 11 18:59:30 dist-test-slave-2j7r krb5kdc[3829](info): commencing operation
krb5kdc: starting...
W20260811 18:59:32.111887 28370 mini_kdc.cc:121] Time spent starting KDC: real 1.174s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474496157106-28370-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474496157106-28370-0/krb5kdc/test-user.keytab.
Aug 11 18:59:32 dist-test-slave-2j7r krb5kdc[3829](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474772, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:59:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:59:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474496157106-28370-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474496157106-28370-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 18:59:32.230361 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474496157106-28370-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474496157106-28370-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474496157106-28370-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37811
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474496157106-28370-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37811
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:59:32.342813 3859 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:59:32.343082 3859 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:59:32.343149 3859 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:59:32.347144 3859 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:59:32.347214 3859 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:59:32.347247 3859 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:59:32.347282 3859 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:59:32.347317 3859 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:59:32.352643 3859 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474496157106-28370-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474496157106-28370-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37811
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474496157106-28370-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37811
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3859
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474496157106-28370-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:59:32.353744 3859 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:59:32.354702 3859 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:59:32.362942 3867 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:59:32.363062 3859 server_base.cc:1034] running on GCE node
W20260811 18:59:32.362951 3870 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:32.362954 3874 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:59:32.363775 3859 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:59:32.364970 3859 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:59:32.366160 3859 hybrid_clock.cc:648] HybridClock initialized: now 1786474772366137 us; error 36 us; skew 500 ppm
Aug 11 18:59:32 dist-test-slave-2j7r krb5kdc[3829](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474772, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:59:32.369124 3859 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 18:59:32.370013 3859 webserver.cc:460] Webserver started at http://127.0.0.1:34777/ using document root <none> and password file <none>
I20260811 18:59:32.370554 3859 fs_manager.cc:359] Metadata directory not provided
I20260811 18:59:32.370618 3859 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:59:32.370810 3859 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:59:32.372797 3859 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474496157106-28370-0/minicluster-data/master-0/data/instance:
uuid: "f7e803b503014e85b903138f8ea2d77b"
format_stamp: "Formatted at 2026-08-11 18:59:32 on dist-test-slave-2j7r"
server_key: "3488d459164ad3fa01f42d194d2e568e"
server_key_iv: "e8340cf127f3971e966b12764e396e80"
server_key_version: "encryptionkey@0"
I20260811 18:59:32.373360 3859 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474496157106-28370-0/minicluster-data/master-0/wal/instance:
uuid: "f7e803b503014e85b903138f8ea2d77b"
format_stamp: "Formatted at 2026-08-11 18:59:32 on dist-test-slave-2j7r"
server_key: "3488d459164ad3fa01f42d194d2e568e"
server_key_iv: "e8340cf127f3971e966b12764e396e80"
server_key_version: "encryptionkey@0"
I20260811 18:59:32.376989 3859 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260811 18:59:32.379293 3887 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:32.380481 3859 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 18:59:32.380594 3859 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474496157106-28370-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474496157106-28370-0/minicluster-data/master-0/wal
uuid: "f7e803b503014e85b903138f8ea2d77b"
format_stamp: "Formatted at 2026-08-11 18:59:32 on dist-test-slave-2j7r"
server_key: "3488d459164ad3fa01f42d194d2e568e"
server_key_iv: "e8340cf127f3971e966b12764e396e80"
server_key_version: "encryptionkey@0"
I20260811 18:59:32.380682 3859 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474496157106-28370-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474496157106-28370-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474496157106-28370-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:59:32.384969 3859 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:59:32.387645 3859 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:59:32.387797 3859 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:59:32.394505 3859 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37811
I20260811 18:59:32.394518 4007 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37811 every 8 connection(s)
I20260811 18:59:32.395337 3859 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
I20260811 18:59:32.396214 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 3859
I20260811 18:59:32.396345 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786474496157106-28370-0/minicluster-data/master-0/wal/instance
I20260811 18:59:32.396588 28370 external_mini_cluster.cc:1428] Setting key 1ea2fe733c60f9d02bde073367047ca4
I20260811 18:59:32.398301 4008 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:59:32.401865 4017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:32.397869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.230:36263 (local address 127.0.0.1:37811)
0811 18:59:32.398315 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:32.398335 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:32.398379 (+ 44us) server_negotiation.cc:407] Connection header received
0811 18:59:32.399173 (+ 794us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:32.399194 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:32.399540 (+ 346us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:32.399896 (+ 356us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:32.400507 (+ 611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:32.400513 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:32.400526 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:59:32.400536 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:32.400591 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:59:32.400619 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:32.400748 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:32.400817 (+ 69us) server_negotiation.cc:299] Negotiation successful
0811 18:59:32.400963 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":111,"threads_started":1}
I20260811 18:59:32.404711 28370 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:59:32.405117 4008 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f7e803b503014e85b903138f8ea2d77b: Bootstrap starting.
I20260811 18:59:32.407617 4008 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f7e803b503014e85b903138f8ea2d77b: Neither blocks nor log segments found. Creating new log.
I20260811 18:59:32.408391 4008 log.cc:824] T 00000000000000000000000000000000 P f7e803b503014e85b903138f8ea2d77b: Log is configured to *not* fsync() on all Append() calls
I20260811 18:59:32.408507 4017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:32.407004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.230:38853 (local address 127.0.0.1:37811)
0811 18:59:32.407113 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:32.407118 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:32.407291 (+ 173us) server_negotiation.cc:407] Connection header received
0811 18:59:32.407409 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:32.407412 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:32.407441 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:32.407499 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:32.408135 (+ 636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:32.408138 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:32.408141 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:59:32.408152 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:32.408177 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:59:32.408187 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:32.408255 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:32.408327 (+ 72us) server_negotiation.cc:299] Negotiation successful
0811 18:59:32.408364 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 18:59:32.410336 4008 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f7e803b503014e85b903138f8ea2d77b: No bootstrap required, opened a new log
I20260811 18:59:32.413754 4008 raft_consensus.cc:348] T 00000000000000000000000000000000 P f7e803b503014e85b903138f8ea2d77b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7e803b503014e85b903138f8ea2d77b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37811 } }
I20260811 18:59:32.413978 4008 raft_consensus.cc:374] T 00000000000000000000000000000000 P f7e803b503014e85b903138f8ea2d77b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:59:32.414018 4008 raft_consensus.cc:729] T 00000000000000000000000000000000 P f7e803b503014e85b903138f8ea2d77b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7e803b503014e85b903138f8ea2d77b, State: Initialized, Role: FOLLOWER
I20260811 18:59:32.414390 4008 consensus_queue.cc:260] T 00000000000000000000000000000000 P f7e803b503014e85b903138f8ea2d77b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7e803b503014e85b903138f8ea2d77b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37811 } }
I20260811 18:59:32.414515 4008 raft_consensus.cc:388] T 00000000000000000000000000000000 P f7e803b503014e85b903138f8ea2d77b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:59:32.414562 4008 raft_consensus.cc:482] T 00000000000000000000000000000000 P f7e803b503014e85b903138f8ea2d77b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:59:32.414631 4008 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f7e803b503014e85b903138f8ea2d77b [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:59:32.415418 4008 raft_consensus.cc:504] T 00000000000000000000000000000000 P f7e803b503014e85b903138f8ea2d77b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7e803b503014e85b903138f8ea2d77b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37811 } }
I20260811 18:59:32.415704 4008 leader_election.cc:302] T 00000000000000000000000000000000 P f7e803b503014e85b903138f8ea2d77b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f7e803b503014e85b903138f8ea2d77b; no voters:
I20260811 18:59:32.415985 4008 leader_election.cc:288] T 00000000000000000000000000000000 P f7e803b503014e85b903138f8ea2d77b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:59:32.416111 4029 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f7e803b503014e85b903138f8ea2d77b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:59:32.416343 4029 raft_consensus.cc:686] T 00000000000000000000000000000000 P f7e803b503014e85b903138f8ea2d77b [term 1 LEADER]: Becoming Leader. State: Replica: f7e803b503014e85b903138f8ea2d77b, State: Running, Role: LEADER
I20260811 18:59:32.416891 4029 consensus_queue.cc:237] T 00000000000000000000000000000000 P f7e803b503014e85b903138f8ea2d77b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7e803b503014e85b903138f8ea2d77b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37811 } }
I20260811 18:59:32.417196 4008 sys_catalog.cc:564] T 00000000000000000000000000000000 P f7e803b503014e85b903138f8ea2d77b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:59:32.418640 4031 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7e803b503014e85b903138f8ea2d77b [sys.catalog]: SysCatalogTable state changed. Reason: New leader f7e803b503014e85b903138f8ea2d77b. Latest consensus state: current_term: 1 leader_uuid: "f7e803b503014e85b903138f8ea2d77b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7e803b503014e85b903138f8ea2d77b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37811 } } }
I20260811 18:59:32.418691 4030 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7e803b503014e85b903138f8ea2d77b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f7e803b503014e85b903138f8ea2d77b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7e803b503014e85b903138f8ea2d77b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37811 } } }
I20260811 18:59:32.418787 4031 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7e803b503014e85b903138f8ea2d77b [sys.catalog]: This master's current role is: LEADER
I20260811 18:59:32.418797 4030 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7e803b503014e85b903138f8ea2d77b [sys.catalog]: This master's current role is: LEADER
I20260811 18:59:32.419204 4056 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:59:32.422431 4056 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:59:32.427523 4056 catalog_manager.cc:1347] Generated new cluster ID: 1a0ebf514668406cb673d493c1761c25
I20260811 18:59:32.427594 4056 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:59:32.451715 4056 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:59:32.452059 4056 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:59:32.458686 4056 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f7e803b503014e85b903138f8ea2d77b: Generated new TSK 0
I20260811 18:59:32.459329 4056 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:59:32.471350 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 3859
2026-08-11T18:59:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1562 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474496157106-28370-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:59:32 dist-test-slave-2j7r krb5kdc[4065](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:59:32 dist-test-slave-2j7r krb5kdc[4065](info): set up 2 sockets
Aug 11 18:59:32 dist-test-slave-2j7r krb5kdc[4065](info): commencing operation
krb5kdc: starting...
W20260811 18:59:33.682052 28370 mini_kdc.cc:121] Time spent starting KDC: real 1.182s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474496157106-28370-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474496157106-28370-0/krb5kdc/test-user.keytab.
Aug 11 18:59:33 dist-test-slave-2j7r krb5kdc[4065](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474773, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:59:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:59:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.180.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.180.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.180.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab.
Entry for principal kudu/127.27.180.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab.
I20260811 18:59:33.801581 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474496157106-28370-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474496157106-28370-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474496157106-28370-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.190:38955
--webserver_interface=127.27.180.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab
--principal=kudu/127.27.180.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:43921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.180.190:38955
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:59:33.908203 4091 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:59:33.908442 4091 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:59:33.908483 4091 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:59:33.912381 4091 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:59:33.912441 4091 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:59:33.912459 4091 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:59:33.912474 4091 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:59:33.912487 4091 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:59:33.917647 4091 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:43921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474496157106-28370-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474496157106-28370-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.180.190:38955
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab
--principal=kudu/127.27.180.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.190:38955
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.180.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4091
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474496157106-28370-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:59:33.918627 4091 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:59:33.919588 4091 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:59:33.927839 4101 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:59:33.927898 4091 server_base.cc:1034] running on GCE node
W20260811 18:59:33.927847 4108 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:33.927858 4103 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:59:33.928610 4091 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:59:33.929718 4091 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:59:33.930892 4091 hybrid_clock.cc:648] HybridClock initialized: now 1786474773930876 us; error 28 us; skew 500 ppm
Aug 11 18:59:33 dist-test-slave-2j7r krb5kdc[4065](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474773, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:59:33.933606 4091 init.cc:377] Logged in from keytab as kudu/127.27.180.190@KRBTEST.COM (short username kudu)
I20260811 18:59:33.934475 4091 webserver.cc:460] Webserver started at http://127.27.180.190:35563/ using document root <none> and password file <none>
I20260811 18:59:33.934964 4091 fs_manager.cc:359] Metadata directory not provided
I20260811 18:59:33.935021 4091 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:59:33.935181 4091 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:59:33.937136 4091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474496157106-28370-0/minicluster-data/master-0/data/instance:
uuid: "8870a32960a84f2690b5472133cce0fd"
format_stamp: "Formatted at 2026-08-11 18:59:33 on dist-test-slave-2j7r"
server_key: "a68b9d8eceac028884709dc05c02d20f"
server_key_iv: "6e1e9ba3f3f434a50439f48152a23b00"
server_key_version: "encryptionkey@0"
I20260811 18:59:33.937667 4091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474496157106-28370-0/minicluster-data/master-0/wal/instance:
uuid: "8870a32960a84f2690b5472133cce0fd"
format_stamp: "Formatted at 2026-08-11 18:59:33 on dist-test-slave-2j7r"
server_key: "a68b9d8eceac028884709dc05c02d20f"
server_key_iv: "6e1e9ba3f3f434a50439f48152a23b00"
server_key_version: "encryptionkey@0"
I20260811 18:59:33.941218 4091 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:59:33.943665 4127 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:33.944965 4091 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 18:59:33.945116 4091 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474496157106-28370-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474496157106-28370-0/minicluster-data/master-0/wal
uuid: "8870a32960a84f2690b5472133cce0fd"
format_stamp: "Formatted at 2026-08-11 18:59:33 on dist-test-slave-2j7r"
server_key: "a68b9d8eceac028884709dc05c02d20f"
server_key_iv: "6e1e9ba3f3f434a50439f48152a23b00"
server_key_version: "encryptionkey@0"
I20260811 18:59:33.945221 4091 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474496157106-28370-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474496157106-28370-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474496157106-28370-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:59:33.957166 4091 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:59:33.960150 4091 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:59:33.960327 4091 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:59:33.967226 4091 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.190:38955
I20260811 18:59:33.967278 4230 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.190:38955 every 8 connection(s)
I20260811 18:59:33.968139 4091 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
I20260811 18:59:33.970942 4231 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:59:33.976521 4231 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8870a32960a84f2690b5472133cce0fd: Bootstrap starting.
I20260811 18:59:33.977236 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 4091
I20260811 18:59:33.977392 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786474496157106-28370-0/minicluster-data/master-0/wal/instance
I20260811 18:59:33.977703 28370 external_mini_cluster.cc:1428] Setting key 8ca1b7a4e48628a2ae5ab7ea7628f825
I20260811 18:59:33.979454 4231 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8870a32960a84f2690b5472133cce0fd: Neither blocks nor log segments found. Creating new log.
I20260811 18:59:33.980157 4231 log.cc:824] T 00000000000000000000000000000000 P 8870a32960a84f2690b5472133cce0fd: Log is configured to *not* fsync() on all Append() calls
I20260811 18:59:33.982385 4231 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8870a32960a84f2690b5472133cce0fd: No bootstrap required, opened a new log
Aug 11 18:59:33 dist-test-slave-2j7r krb5kdc[4065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474773, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM
I20260811 18:59:33.986692 4231 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8870a32960a84f2690b5472133cce0fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8870a32960a84f2690b5472133cce0fd" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 38955 } }
I20260811 18:59:33.986912 4231 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8870a32960a84f2690b5472133cce0fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:59:33.986961 4231 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8870a32960a84f2690b5472133cce0fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8870a32960a84f2690b5472133cce0fd, State: Initialized, Role: FOLLOWER
I20260811 18:59:33.987406 4231 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8870a32960a84f2690b5472133cce0fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8870a32960a84f2690b5472133cce0fd" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 38955 } }
I20260811 18:59:33.987558 4231 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8870a32960a84f2690b5472133cce0fd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:59:33.987663 4231 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8870a32960a84f2690b5472133cce0fd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:59:33.987793 4231 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8870a32960a84f2690b5472133cce0fd [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:59:33.988916 4231 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8870a32960a84f2690b5472133cce0fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8870a32960a84f2690b5472133cce0fd" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 38955 } }
I20260811 18:59:33.989303 4231 leader_election.cc:302] T 00000000000000000000000000000000 P 8870a32960a84f2690b5472133cce0fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8870a32960a84f2690b5472133cce0fd; no voters:
I20260811 18:59:33.989609 4231 leader_election.cc:288] T 00000000000000000000000000000000 P 8870a32960a84f2690b5472133cce0fd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:59:33.989711 4239 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8870a32960a84f2690b5472133cce0fd [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:59:33.989959 4239 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8870a32960a84f2690b5472133cce0fd [term 1 LEADER]: Becoming Leader. State: Replica: 8870a32960a84f2690b5472133cce0fd, State: Running, Role: LEADER
I20260811 18:59:33.990307 4239 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8870a32960a84f2690b5472133cce0fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8870a32960a84f2690b5472133cce0fd" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 38955 } }
I20260811 18:59:33.990960 4231 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8870a32960a84f2690b5472133cce0fd [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:59:33.992105 4240 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8870a32960a84f2690b5472133cce0fd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8870a32960a84f2690b5472133cce0fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8870a32960a84f2690b5472133cce0fd" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 38955 } } }
I20260811 18:59:33.992228 4240 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8870a32960a84f2690b5472133cce0fd [sys.catalog]: This master's current role is: LEADER
I20260811 18:59:33.992604 4241 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8870a32960a84f2690b5472133cce0fd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8870a32960a84f2690b5472133cce0fd. Latest consensus state: current_term: 1 leader_uuid: "8870a32960a84f2690b5472133cce0fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8870a32960a84f2690b5472133cce0fd" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 38955 } } }
I20260811 18:59:33.992650 4237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:33.978916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54919 (local address 127.27.180.190:38955)
0811 18:59:33.979354 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:33.979370 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:33.979405 (+ 35us) server_negotiation.cc:407] Connection header received
0811 18:59:33.980055 (+ 650us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:33.980072 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:33.980428 (+ 356us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:33.980775 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:33.981705 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:33.982699 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:33.983834 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:33.984274 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:33.984387 (+ 113us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:33.987183 (+ 2796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:33.987188 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:33.987201 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:59:33.987227 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:33.989135 (+ 1908us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:33.989666 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:33.989673 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:33.989679 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:33.989820 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:33.990254 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:33.990257 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:33.990258 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:33.990731 (+ 473us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:33.990935 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:33.991399 (+ 464us) server_negotiation.cc:299] Negotiation successful
0811 18:59:33.991628 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295,"thread_start_us":126,"threads_started":1}
I20260811 18:59:33.993377 4241 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8870a32960a84f2690b5472133cce0fd [sys.catalog]: This master's current role is: LEADER
W20260811 18:59:33.995893 4271 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8870a32960a84f2690b5472133cce0fd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:59:33.995996 4271 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:59:33.996076 28370 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:59:33.996119 4261 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:59:33.997054 4261 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:59:34.002426 4261 catalog_manager.cc:1347] Generated new cluster ID: 1a403a9dfb7f40c6a4ae3058d30a65cb
I20260811 18:59:34.002503 4261 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:59:34.007972 4237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:33.997948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36295 (local address 127.27.180.190:38955)
0811 18:59:33.998063 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:33.998067 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:34.000551 (+ 2484us) server_negotiation.cc:407] Connection header received
0811 18:59:34.000659 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:34.000662 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:34.000707 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:34.000792 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:34.001607 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:34.002361 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:34.003301 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:34.003555 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:34.003643 (+ 88us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:34.004937 (+ 1294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:34.004940 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:34.004942 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:59:34.004963 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:34.006284 (+ 1321us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:34.006714 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:34.006717 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:34.006718 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:34.006785 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:34.007164 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:34.007167 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:34.007169 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:34.007332 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:34.007468 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:34.007713 (+ 245us) server_negotiation.cc:299] Negotiation successful
0811 18:59:34.007809 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 18:59:34.025225 4261 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:59:34.026173 4261 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:59:34.031761 4261 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8870a32960a84f2690b5472133cce0fd: Generated new TSK 0
I20260811 18:59:34.032423 4261 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:59:34.044894 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 4091
W20260811 18:59:34.064564 4063 connection.cc:535] client connection to 127.27.180.190:38955 recv error: Network error: failed to read from TLS socket (remote: 127.27.180.190:38955): Connection reset by peer (error 104)
2026-08-11T18:59:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1579 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474496157106-28370-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:59:34 dist-test-slave-2j7r krb5kdc[4291](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:59:34 dist-test-slave-2j7r krb5kdc[4291](info): set up 2 sockets
Aug 11 18:59:34 dist-test-slave-2j7r krb5kdc[4291](info): commencing operation
krb5kdc: starting...
W20260811 18:59:35.249347 28370 mini_kdc.cc:121] Time spent starting KDC: real 1.171s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474496157106-28370-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474496157106-28370-0/krb5kdc/test-user.keytab.
Aug 11 18:59:35 dist-test-slave-2j7r krb5kdc[4291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474775, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:59:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:59:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.180.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.180.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.180.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab.
Entry for principal kudu/127.27.180.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab.
I20260811 18:59:35.367667 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474496157106-28370-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474496157106-28370-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474496157106-28370-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.190:33321
--webserver_interface=127.27.180.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab
--principal=kudu/127.27.180.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:33587
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.180.190:33321
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:59:35.476190 4314 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:59:35.476436 4314 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:59:35.476491 4314 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:59:35.480423 4314 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:59:35.480482 4314 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:59:35.480508 4314 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:59:35.480530 4314 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:59:35.480558 4314 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:59:35.485666 4314 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:33587
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474496157106-28370-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474496157106-28370-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.180.190:33321
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab
--principal=kudu/127.27.180.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.190:33321
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.180.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4314
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474496157106-28370-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:59:35.486658 4314 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:59:35.487622 4314 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:59:35.495715 4322 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:35.495779 4327 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:35.495721 4323 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:59:35.495934 4314 server_base.cc:1034] running on GCE node
I20260811 18:59:35.496603 4314 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:59:35.497663 4314 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:59:35.498826 4314 hybrid_clock.cc:648] HybridClock initialized: now 1786474775498806 us; error 31 us; skew 500 ppm
Aug 11 18:59:35 dist-test-slave-2j7r krb5kdc[4291](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474775, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:59:35.501380 4314 init.cc:377] Logged in from keytab as kudu/127.27.180.190@KRBTEST.COM (short username kudu)
I20260811 18:59:35.502168 4314 webserver.cc:460] Webserver started at http://127.27.180.190:33217/ using document root <none> and password file <none>
I20260811 18:59:35.502672 4314 fs_manager.cc:359] Metadata directory not provided
I20260811 18:59:35.502735 4314 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:59:35.502929 4314 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:59:35.504906 4314 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474496157106-28370-0/minicluster-data/master-0/data/instance:
uuid: "1adba401e3f4479cb8095b5918f74adf"
format_stamp: "Formatted at 2026-08-11 18:59:35 on dist-test-slave-2j7r"
server_key: "5b585ec8045d3044aac958fea26ed458"
server_key_iv: "fc85673561277ba7be5a8d8f0657c784"
server_key_version: "encryptionkey@0"
I20260811 18:59:35.505487 4314 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474496157106-28370-0/minicluster-data/master-0/wal/instance:
uuid: "1adba401e3f4479cb8095b5918f74adf"
format_stamp: "Formatted at 2026-08-11 18:59:35 on dist-test-slave-2j7r"
server_key: "5b585ec8045d3044aac958fea26ed458"
server_key_iv: "fc85673561277ba7be5a8d8f0657c784"
server_key_version: "encryptionkey@0"
I20260811 18:59:35.508988 4314 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 18:59:35.511430 4345 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:35.512588 4314 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 18:59:35.512717 4314 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474496157106-28370-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474496157106-28370-0/minicluster-data/master-0/wal
uuid: "1adba401e3f4479cb8095b5918f74adf"
format_stamp: "Formatted at 2026-08-11 18:59:35 on dist-test-slave-2j7r"
server_key: "5b585ec8045d3044aac958fea26ed458"
server_key_iv: "fc85673561277ba7be5a8d8f0657c784"
server_key_version: "encryptionkey@0"
I20260811 18:59:35.512818 4314 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474496157106-28370-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474496157106-28370-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474496157106-28370-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:59:35.542826 4314 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:59:35.546197 4314 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:59:35.546371 4314 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:59:35.553390 4314 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.190:33321
I20260811 18:59:35.553411 4427 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.190:33321 every 8 connection(s)
I20260811 18:59:35.554354 4314 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
I20260811 18:59:35.557013 4428 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:59:35.562431 4428 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1adba401e3f4479cb8095b5918f74adf: Bootstrap starting.
I20260811 18:59:35.563900 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 4314
I20260811 18:59:35.564026 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786474496157106-28370-0/minicluster-data/master-0/wal/instance
I20260811 18:59:35.564271 28370 external_mini_cluster.cc:1428] Setting key 717274e22e771a6e80e372d48844fe72
I20260811 18:59:35.565011 4428 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1adba401e3f4479cb8095b5918f74adf: Neither blocks nor log segments found. Creating new log.
I20260811 18:59:35.565788 4428 log.cc:824] T 00000000000000000000000000000000 P 1adba401e3f4479cb8095b5918f74adf: Log is configured to *not* fsync() on all Append() calls
I20260811 18:59:35.567689 4428 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1adba401e3f4479cb8095b5918f74adf: No bootstrap required, opened a new log
I20260811 18:59:35.571708 4428 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1adba401e3f4479cb8095b5918f74adf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1adba401e3f4479cb8095b5918f74adf" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 33321 } }
I20260811 18:59:35.571974 4428 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1adba401e3f4479cb8095b5918f74adf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:59:35.572091 4428 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1adba401e3f4479cb8095b5918f74adf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1adba401e3f4479cb8095b5918f74adf, State: Initialized, Role: FOLLOWER
I20260811 18:59:35.572523 4428 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1adba401e3f4479cb8095b5918f74adf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1adba401e3f4479cb8095b5918f74adf" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 33321 } }
I20260811 18:59:35.572642 4428 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1adba401e3f4479cb8095b5918f74adf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:59:35.572700 4428 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1adba401e3f4479cb8095b5918f74adf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:59:35.572785 4428 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1adba401e3f4479cb8095b5918f74adf [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:59:35.572897 4434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:35.565364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59729 (local address 127.27.180.190:33321)
0811 18:59:35.565829 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:35.565845 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:35.565899 (+ 54us) server_negotiation.cc:407] Connection header received
0811 18:59:35.566532 (+ 633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:35.566548 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:35.566850 (+ 302us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:35.567116 (+ 266us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:35.567465 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:35.568446 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:35.569769 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:35.570227 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:35.570345 (+ 118us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:35.571209 (+ 864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:35.571214 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:35.571226 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:59:35.571584 (+ 358us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:35.571639 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:59:35.571661 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:35.571759 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:35.571947 (+ 188us) server_negotiation.cc:299] Negotiation successful
0811 18:59:35.572081 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":138,"threads_started":1}
I20260811 18:59:35.573633 4428 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1adba401e3f4479cb8095b5918f74adf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1adba401e3f4479cb8095b5918f74adf" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 33321 } }
I20260811 18:59:35.573940 4428 leader_election.cc:302] T 00000000000000000000000000000000 P 1adba401e3f4479cb8095b5918f74adf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1adba401e3f4479cb8095b5918f74adf; no voters:
I20260811 18:59:35.574200 4428 leader_election.cc:288] T 00000000000000000000000000000000 P 1adba401e3f4479cb8095b5918f74adf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:59:35.574285 4436 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1adba401e3f4479cb8095b5918f74adf [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:59:35.574941 4436 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1adba401e3f4479cb8095b5918f74adf [term 1 LEADER]: Becoming Leader. State: Replica: 1adba401e3f4479cb8095b5918f74adf, State: Running, Role: LEADER
I20260811 18:59:35.575287 4436 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1adba401e3f4479cb8095b5918f74adf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1adba401e3f4479cb8095b5918f74adf" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 33321 } }
I20260811 18:59:35.575338 4428 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1adba401e3f4479cb8095b5918f74adf [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:59:35.575619 28370 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:59:35.577037 4437 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1adba401e3f4479cb8095b5918f74adf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1adba401e3f4479cb8095b5918f74adf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1adba401e3f4479cb8095b5918f74adf" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 33321 } } }
I20260811 18:59:35.577189 4437 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1adba401e3f4479cb8095b5918f74adf [sys.catalog]: This master's current role is: LEADER
I20260811 18:59:35.577519 4477 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:59:35.577490 4439 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1adba401e3f4479cb8095b5918f74adf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1adba401e3f4479cb8095b5918f74adf. Latest consensus state: current_term: 1 leader_uuid: "1adba401e3f4479cb8095b5918f74adf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1adba401e3f4479cb8095b5918f74adf" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 33321 } } }
I20260811 18:59:35.577934 4439 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1adba401e3f4479cb8095b5918f74adf [sys.catalog]: This master's current role is: LEADER
I20260811 18:59:35.580646 4477 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:59:35.583393 4434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:35.578040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43537 (local address 127.27.180.190:33321)
0811 18:59:35.578127 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:35.578131 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:35.579065 (+ 934us) server_negotiation.cc:407] Connection header received
0811 18:59:35.579403 (+ 338us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:35.579406 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:35.579448 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:35.579494 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:35.579876 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:35.580578 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:35.581718 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:35.582038 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:35.582141 (+ 103us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:35.582959 (+ 818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:35.582963 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:35.582966 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:59:35.582978 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:35.583004 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:59:35.583015 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:35.583101 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:35.583235 (+ 134us) server_negotiation.cc:299] Negotiation successful
0811 18:59:35.583266 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260811 18:59:35.585709 4477 catalog_manager.cc:1347] Generated new cluster ID: 4dbbd0fcbf9243d3af1c46149291fcaa
I20260811 18:59:35.585770 4477 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:59:35.612149 4477 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:59:35.613227 4477 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:59:35.617403 4477 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1adba401e3f4479cb8095b5918f74adf: Generated new TSK 0
I20260811 18:59:35.618105 4477 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:59:35.624274 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 4314
W20260811 18:59:35.641665 4286 connection.cc:535] client connection to 127.27.180.190:33321 recv error: Network error: failed to read from TLS socket (remote: 127.27.180.190:33321): Connection reset by peer (error 104)
2026-08-11T18:59:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1577 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474496157106-28370-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:59:35 dist-test-slave-2j7r krb5kdc[4495](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:59:35 dist-test-slave-2j7r krb5kdc[4495](info): set up 2 sockets
Aug 11 18:59:35 dist-test-slave-2j7r krb5kdc[4495](info): commencing operation
krb5kdc: starting...
W20260811 18:59:36.824781 28370 mini_kdc.cc:121] Time spent starting KDC: real 1.169s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474496157106-28370-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474496157106-28370-0/krb5kdc/test-user.keytab.
Aug 11 18:59:36 dist-test-slave-2j7r krb5kdc[4495](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474776, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:59:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:59:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.180.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.180.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.180.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab.
Entry for principal kudu/127.27.180.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab.
I20260811 18:59:36.944307 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474496157106-28370-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474496157106-28370-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474496157106-28370-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.190:41067
--webserver_interface=127.27.180.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab
--principal=kudu/127.27.180.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:44683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.180.190:41067
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:59:37.054551 4516 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:59:37.054813 4516 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:59:37.054868 4516 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:59:37.058851 4516 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:59:37.058915 4516 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:59:37.058941 4516 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:59:37.058961 4516 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:59:37.058987 4516 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:59:37.064467 4516 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:44683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474496157106-28370-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474496157106-28370-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.180.190:41067
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab
--principal=kudu/127.27.180.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.190:41067
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.180.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4516
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474496157106-28370-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:59:37.065548 4516 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:59:37.066558 4516 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:59:37.074831 4525 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:37.074821 4531 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:37.074865 4528 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:59:37.075047 4516 server_base.cc:1034] running on GCE node
I20260811 18:59:37.075575 4516 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:59:37.076611 4516 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:59:37.077762 4516 hybrid_clock.cc:648] HybridClock initialized: now 1786474777077744 us; error 31 us; skew 500 ppm
Aug 11 18:59:37 dist-test-slave-2j7r krb5kdc[4495](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474777, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:59:37.080329 4516 init.cc:377] Logged in from keytab as kudu/127.27.180.190@KRBTEST.COM (short username kudu)
I20260811 18:59:37.081202 4516 webserver.cc:460] Webserver started at http://127.27.180.190:36497/ using document root <none> and password file <none>
I20260811 18:59:37.081704 4516 fs_manager.cc:359] Metadata directory not provided
I20260811 18:59:37.081768 4516 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:59:37.081971 4516 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:59:37.083868 4516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474496157106-28370-0/minicluster-data/master-0/data/instance:
uuid: "f13d181ff8af4ff3a0f9d5fae29fd589"
format_stamp: "Formatted at 2026-08-11 18:59:37 on dist-test-slave-2j7r"
server_key: "a70641e89a985593eee22a1ad9256781"
server_key_iv: "ebe5b353ec24824c5309a4fc57045449"
server_key_version: "encryptionkey@0"
I20260811 18:59:37.084405 4516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474496157106-28370-0/minicluster-data/master-0/wal/instance:
uuid: "f13d181ff8af4ff3a0f9d5fae29fd589"
format_stamp: "Formatted at 2026-08-11 18:59:37 on dist-test-slave-2j7r"
server_key: "a70641e89a985593eee22a1ad9256781"
server_key_iv: "ebe5b353ec24824c5309a4fc57045449"
server_key_version: "encryptionkey@0"
I20260811 18:59:37.088011 4516 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260811 18:59:37.090283 4551 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:37.091445 4516 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 18:59:37.091555 4516 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474496157106-28370-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474496157106-28370-0/minicluster-data/master-0/wal
uuid: "f13d181ff8af4ff3a0f9d5fae29fd589"
format_stamp: "Formatted at 2026-08-11 18:59:37 on dist-test-slave-2j7r"
server_key: "a70641e89a985593eee22a1ad9256781"
server_key_iv: "ebe5b353ec24824c5309a4fc57045449"
server_key_version: "encryptionkey@0"
I20260811 18:59:37.091645 4516 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474496157106-28370-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474496157106-28370-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474496157106-28370-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:59:37.095932 4516 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:59:37.098489 4516 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:59:37.098640 4516 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:59:37.105010 4516 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.190:41067
I20260811 18:59:37.105036 4665 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.190:41067 every 8 connection(s)
I20260811 18:59:37.105867 4516 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
I20260811 18:59:37.108456 4666 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:59:37.109915 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 4516
I20260811 18:59:37.110045 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786474496157106-28370-0/minicluster-data/master-0/wal/instance
I20260811 18:59:37.110283 28370 external_mini_cluster.cc:1428] Setting key 8d2c6bc2b0b27fb9c4c80030f30f4dab
I20260811 18:59:37.114724 4666 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f13d181ff8af4ff3a0f9d5fae29fd589: Bootstrap starting.
I20260811 18:59:37.115417 4670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:37.111557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38713 (local address 127.27.180.190:41067)
0811 18:59:37.112008 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:37.112028 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:37.112079 (+ 51us) server_negotiation.cc:407] Connection header received
0811 18:59:37.112944 (+ 865us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:37.112966 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:37.113265 (+ 299us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:37.113525 (+ 260us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:37.114285 (+ 760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:37.114290 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:37.114301 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:59:37.114311 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:37.114359 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:59:37.114381 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:37.114485 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:37.114574 (+ 89us) server_negotiation.cc:299] Negotiation successful
0811 18:59:37.114717 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":100,"threads_started":1}
I20260811 18:59:37.117601 4666 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f13d181ff8af4ff3a0f9d5fae29fd589: Neither blocks nor log segments found. Creating new log.
I20260811 18:59:37.117837 28370 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:59:37.118407 4666 log.cc:824] T 00000000000000000000000000000000 P f13d181ff8af4ff3a0f9d5fae29fd589: Log is configured to *not* fsync() on all Append() calls
I20260811 18:59:37.120451 4666 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f13d181ff8af4ff3a0f9d5fae29fd589: No bootstrap required, opened a new log
I20260811 18:59:37.121302 4670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:37.119891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48235 (local address 127.27.180.190:41067)
0811 18:59:37.120002 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:37.120006 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:37.120182 (+ 176us) server_negotiation.cc:407] Connection header received
0811 18:59:37.120261 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:37.120264 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:37.120294 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:37.120371 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:37.120951 (+ 580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:37.120954 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:37.120957 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:59:37.120968 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:37.120993 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:59:37.121006 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:37.121115 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:37.121168 (+ 53us) server_negotiation.cc:299] Negotiation successful
0811 18:59:37.121201 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 18:59:37.123905 4666 raft_consensus.cc:348] T 00000000000000000000000000000000 P f13d181ff8af4ff3a0f9d5fae29fd589 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f13d181ff8af4ff3a0f9d5fae29fd589" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 41067 } }
I20260811 18:59:37.124101 4666 raft_consensus.cc:374] T 00000000000000000000000000000000 P f13d181ff8af4ff3a0f9d5fae29fd589 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:59:37.124132 4666 raft_consensus.cc:729] T 00000000000000000000000000000000 P f13d181ff8af4ff3a0f9d5fae29fd589 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f13d181ff8af4ff3a0f9d5fae29fd589, State: Initialized, Role: FOLLOWER
I20260811 18:59:37.124469 4666 consensus_queue.cc:260] T 00000000000000000000000000000000 P f13d181ff8af4ff3a0f9d5fae29fd589 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f13d181ff8af4ff3a0f9d5fae29fd589" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 41067 } }
I20260811 18:59:37.124577 4666 raft_consensus.cc:388] T 00000000000000000000000000000000 P f13d181ff8af4ff3a0f9d5fae29fd589 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:59:37.124665 4666 raft_consensus.cc:482] T 00000000000000000000000000000000 P f13d181ff8af4ff3a0f9d5fae29fd589 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:59:37.124768 4666 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f13d181ff8af4ff3a0f9d5fae29fd589 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:59:37.125617 4666 raft_consensus.cc:504] T 00000000000000000000000000000000 P f13d181ff8af4ff3a0f9d5fae29fd589 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f13d181ff8af4ff3a0f9d5fae29fd589" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 41067 } }
I20260811 18:59:37.125914 4666 leader_election.cc:302] T 00000000000000000000000000000000 P f13d181ff8af4ff3a0f9d5fae29fd589 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f13d181ff8af4ff3a0f9d5fae29fd589; no voters:
I20260811 18:59:37.126191 4666 leader_election.cc:288] T 00000000000000000000000000000000 P f13d181ff8af4ff3a0f9d5fae29fd589 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:59:37.126284 4684 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f13d181ff8af4ff3a0f9d5fae29fd589 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:59:37.126494 4684 raft_consensus.cc:686] T 00000000000000000000000000000000 P f13d181ff8af4ff3a0f9d5fae29fd589 [term 1 LEADER]: Becoming Leader. State: Replica: f13d181ff8af4ff3a0f9d5fae29fd589, State: Running, Role: LEADER
I20260811 18:59:37.126816 4684 consensus_queue.cc:237] T 00000000000000000000000000000000 P f13d181ff8af4ff3a0f9d5fae29fd589 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f13d181ff8af4ff3a0f9d5fae29fd589" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 41067 } }
I20260811 18:59:37.127226 4666 sys_catalog.cc:564] T 00000000000000000000000000000000 P f13d181ff8af4ff3a0f9d5fae29fd589 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:59:37.128381 4687 sys_catalog.cc:455] T 00000000000000000000000000000000 P f13d181ff8af4ff3a0f9d5fae29fd589 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f13d181ff8af4ff3a0f9d5fae29fd589" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f13d181ff8af4ff3a0f9d5fae29fd589" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 41067 } } }
I20260811 18:59:37.128481 4687 sys_catalog.cc:458] T 00000000000000000000000000000000 P f13d181ff8af4ff3a0f9d5fae29fd589 [sys.catalog]: This master's current role is: LEADER
I20260811 18:59:37.128722 4689 sys_catalog.cc:455] T 00000000000000000000000000000000 P f13d181ff8af4ff3a0f9d5fae29fd589 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f13d181ff8af4ff3a0f9d5fae29fd589. Latest consensus state: current_term: 1 leader_uuid: "f13d181ff8af4ff3a0f9d5fae29fd589" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f13d181ff8af4ff3a0f9d5fae29fd589" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 41067 } } }
I20260811 18:59:37.128811 4689 sys_catalog.cc:458] T 00000000000000000000000000000000 P f13d181ff8af4ff3a0f9d5fae29fd589 [sys.catalog]: This master's current role is: LEADER
I20260811 18:59:37.128825 4699 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:59:37.132021 4699 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:59:37.136754 4699 catalog_manager.cc:1347] Generated new cluster ID: dd6cbb16b5e8427586551008dff15d03
I20260811 18:59:37.136813 4699 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:59:37.153818 4699 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:59:37.154126 4699 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:59:37.162523 4699 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f13d181ff8af4ff3a0f9d5fae29fd589: Generated new TSK 0
I20260811 18:59:37.163262 4699 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:59:37.169018 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 4516
2026-08-11T18:59:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1544 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474496157106-28370-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:59:37 dist-test-slave-2j7r krb5kdc[4719](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:59:37 dist-test-slave-2j7r krb5kdc[4719](info): set up 2 sockets
Aug 11 18:59:37 dist-test-slave-2j7r krb5kdc[4719](info): commencing operation
krb5kdc: starting...
W20260811 18:59:38.369099 28370 mini_kdc.cc:121] Time spent starting KDC: real 1.169s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474496157106-28370-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474496157106-28370-0/krb5kdc/test-user.keytab.
Aug 11 18:59:38 dist-test-slave-2j7r krb5kdc[4719](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474778, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:59:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:59:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.180.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.180.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.180.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab.
Entry for principal kudu/127.27.180.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab.
I20260811 18:59:38.491273 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474496157106-28370-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474496157106-28370-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474496157106-28370-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.190:45723
--webserver_interface=127.27.180.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab
--principal=kudu/127.27.180.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:41917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.180.190:45723
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:59:38.601153 4745 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:59:38.601404 4745 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:59:38.601446 4745 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:59:38.605348 4745 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:59:38.605402 4745 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:59:38.605423 4745 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:59:38.605453 4745 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:59:38.605479 4745 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:59:38.610599 4745 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:41917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474496157106-28370-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474496157106-28370-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.180.190:45723
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab
--principal=kudu/127.27.180.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.190:45723
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.180.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4745
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474496157106-28370-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:59:38.611577 4745 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:59:38.612592 4745 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:59:38.620880 4753 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:38.620909 4757 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:59:38.620991 4745 server_base.cc:1034] running on GCE node
W20260811 18:59:38.620860 4755 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:59:38.621678 4745 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:59:38.622694 4745 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:59:38.623845 4745 hybrid_clock.cc:648] HybridClock initialized: now 1786474778623829 us; error 28 us; skew 500 ppm
Aug 11 18:59:38 dist-test-slave-2j7r krb5kdc[4719](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474778, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:59:38.626461 4745 init.cc:377] Logged in from keytab as kudu/127.27.180.190@KRBTEST.COM (short username kudu)
I20260811 18:59:38.627274 4745 webserver.cc:460] Webserver started at http://127.27.180.190:35147/ using document root <none> and password file <none>
I20260811 18:59:38.627789 4745 fs_manager.cc:359] Metadata directory not provided
I20260811 18:59:38.627848 4745 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:59:38.628046 4745 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:59:38.630089 4745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474496157106-28370-0/minicluster-data/master-0/data/instance:
uuid: "2687576cab5e404e8796d328c41a598d"
format_stamp: "Formatted at 2026-08-11 18:59:38 on dist-test-slave-2j7r"
server_key: "5dc2f29146201ec0bd2d418c6963cfa7"
server_key_iv: "25e8f695d597e2e8546d37b16507065f"
server_key_version: "encryptionkey@0"
I20260811 18:59:38.630651 4745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474496157106-28370-0/minicluster-data/master-0/wal/instance:
uuid: "2687576cab5e404e8796d328c41a598d"
format_stamp: "Formatted at 2026-08-11 18:59:38 on dist-test-slave-2j7r"
server_key: "5dc2f29146201ec0bd2d418c6963cfa7"
server_key_iv: "25e8f695d597e2e8546d37b16507065f"
server_key_version: "encryptionkey@0"
I20260811 18:59:38.634195 4745 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.004s
I20260811 18:59:38.636412 4774 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:38.637635 4745 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 18:59:38.637748 4745 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474496157106-28370-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474496157106-28370-0/minicluster-data/master-0/wal
uuid: "2687576cab5e404e8796d328c41a598d"
format_stamp: "Formatted at 2026-08-11 18:59:38 on dist-test-slave-2j7r"
server_key: "5dc2f29146201ec0bd2d418c6963cfa7"
server_key_iv: "25e8f695d597e2e8546d37b16507065f"
server_key_version: "encryptionkey@0"
I20260811 18:59:38.637837 4745 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474496157106-28370-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474496157106-28370-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474496157106-28370-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:59:38.659122 4745 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:59:38.661726 4745 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:59:38.661911 4745 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:59:38.668960 4745 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.190:45723
I20260811 18:59:38.668994 4863 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.190:45723 every 8 connection(s)
I20260811 18:59:38.669888 4745 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
I20260811 18:59:38.672511 4865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:59:38.677380 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 4745
I20260811 18:59:38.677512 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786474496157106-28370-0/minicluster-data/master-0/wal/instance
I20260811 18:59:38.677752 28370 external_mini_cluster.cc:1428] Setting key 77e8d8bb6c0a34ea97076ba64349e58d
I20260811 18:59:38.678117 4865 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2687576cab5e404e8796d328c41a598d: Bootstrap starting.
I20260811 18:59:38.680594 4865 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2687576cab5e404e8796d328c41a598d: Neither blocks nor log segments found. Creating new log.
I20260811 18:59:38.681421 4865 log.cc:824] T 00000000000000000000000000000000 P 2687576cab5e404e8796d328c41a598d: Log is configured to *not* fsync() on all Append() calls
I20260811 18:59:38.683235 4865 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2687576cab5e404e8796d328c41a598d: No bootstrap required, opened a new log
Aug 11 18:59:38 dist-test-slave-2j7r krb5kdc[4719](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474778, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM
I20260811 18:59:38.687032 4865 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2687576cab5e404e8796d328c41a598d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2687576cab5e404e8796d328c41a598d" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 45723 } }
I20260811 18:59:38.687265 4865 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2687576cab5e404e8796d328c41a598d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:59:38.687326 4865 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2687576cab5e404e8796d328c41a598d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2687576cab5e404e8796d328c41a598d, State: Initialized, Role: FOLLOWER
I20260811 18:59:38.687747 4865 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2687576cab5e404e8796d328c41a598d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2687576cab5e404e8796d328c41a598d" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 45723 } }
I20260811 18:59:38.687906 4865 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2687576cab5e404e8796d328c41a598d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:59:38.687980 4865 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2687576cab5e404e8796d328c41a598d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:59:38.688084 4865 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2687576cab5e404e8796d328c41a598d [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:59:38.688910 4865 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2687576cab5e404e8796d328c41a598d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2687576cab5e404e8796d328c41a598d" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 45723 } }
I20260811 18:59:38.689247 4865 leader_election.cc:302] T 00000000000000000000000000000000 P 2687576cab5e404e8796d328c41a598d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2687576cab5e404e8796d328c41a598d; no voters:
I20260811 18:59:38.689532 4865 leader_election.cc:288] T 00000000000000000000000000000000 P 2687576cab5e404e8796d328c41a598d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:59:38.689616 4873 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2687576cab5e404e8796d328c41a598d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:59:38.689788 4873 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2687576cab5e404e8796d328c41a598d [term 1 LEADER]: Becoming Leader. State: Replica: 2687576cab5e404e8796d328c41a598d, State: Running, Role: LEADER
I20260811 18:59:38.690136 4873 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2687576cab5e404e8796d328c41a598d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2687576cab5e404e8796d328c41a598d" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 45723 } }
I20260811 18:59:38.690819 4865 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2687576cab5e404e8796d328c41a598d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:59:38.691835 4880 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2687576cab5e404e8796d328c41a598d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2687576cab5e404e8796d328c41a598d. Latest consensus state: current_term: 1 leader_uuid: "2687576cab5e404e8796d328c41a598d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2687576cab5e404e8796d328c41a598d" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 45723 } } }
I20260811 18:59:38.691967 4880 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2687576cab5e404e8796d328c41a598d [sys.catalog]: This master's current role is: LEADER
I20260811 18:59:38.692137 4869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:38.679151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35889 (local address 127.27.180.190:45723)
0811 18:59:38.679608 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:38.679624 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:38.679658 (+ 34us) server_negotiation.cc:407] Connection header received
0811 18:59:38.680274 (+ 616us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:38.680289 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:38.680620 (+ 331us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:38.680964 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:38.681803 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:38.682759 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:38.683619 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:38.684082 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:38.684224 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:38.686910 (+ 2686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:38.686915 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:38.686927 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:59:38.686953 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:38.688660 (+ 1707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:38.689061 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:38.689067 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:38.689072 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:38.689224 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:38.689501 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:38.689504 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:38.689506 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:38.690028 (+ 522us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:38.690729 (+ 701us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:38.690846 (+ 117us) server_negotiation.cc:299] Negotiation successful
0811 18:59:38.691124 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":139,"threads_started":1}
I20260811 18:59:38.692277 4874 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2687576cab5e404e8796d328c41a598d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2687576cab5e404e8796d328c41a598d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2687576cab5e404e8796d328c41a598d" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 45723 } } }
I20260811 18:59:38.692360 4874 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2687576cab5e404e8796d328c41a598d [sys.catalog]: This master's current role is: LEADER
I20260811 18:59:38.693185 4891 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:59:38.694955 28370 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:59:38.696192 4891 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:59:38.702661 4891 catalog_manager.cc:1347] Generated new cluster ID: 243085ba83e64121a71b5c317410f02f
I20260811 18:59:38.702776 4891 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:59:38.705605 4869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:38.696800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55117 (local address 127.27.180.190:45723)
0811 18:59:38.696915 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:38.696919 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:38.697314 (+ 395us) server_negotiation.cc:407] Connection header received
0811 18:59:38.697424 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:38.697427 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:38.697468 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:38.697531 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:38.698187 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:38.699014 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:38.700097 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:38.700901 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:38.701022 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:38.702284 (+ 1262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:38.702288 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:38.702291 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:59:38.702316 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:38.703943 (+ 1627us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:38.704424 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:38.704427 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:38.704429 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:38.704501 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:38.704786 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:38.704788 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:38.704790 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:38.704967 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:38.705070 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:38.705354 (+ 284us) server_negotiation.cc:299] Negotiation successful
0811 18:59:38.705462 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 18:59:38.726397 4891 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:59:38.727241 4891 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:59:38.730918 4891 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2687576cab5e404e8796d328c41a598d: Generated new TSK 0
I20260811 18:59:38.731495 4891 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:59:38.740545 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 4745
2026-08-11T18:59:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1566 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474496157106-28370-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:59:38 dist-test-slave-2j7r krb5kdc[4925](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:59:38 dist-test-slave-2j7r krb5kdc[4925](info): set up 2 sockets
Aug 11 18:59:38 dist-test-slave-2j7r krb5kdc[4925](info): commencing operation
krb5kdc: starting...
W20260811 18:59:39.936383 28370 mini_kdc.cc:121] Time spent starting KDC: real 1.169s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474496157106-28370-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474496157106-28370-0/krb5kdc/test-user.keytab.
Aug 11 18:59:40 dist-test-slave-2j7r krb5kdc[4925](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474780, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:59:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:59:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.180.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.180.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.180.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab.
Entry for principal kudu/127.27.180.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab.
I20260811 18:59:40.057696 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474496157106-28370-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474496157106-28370-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474496157106-28370-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.190:44391
--webserver_interface=127.27.180.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab
--principal=kudu/127.27.180.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:33969
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.180.190:44391
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:59:40.166321 4955 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:59:40.166553 4955 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:59:40.166591 4955 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:59:40.170512 4955 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:59:40.170563 4955 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:59:40.170580 4955 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:59:40.170596 4955 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:59:40.170612 4955 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:59:40.175710 4955 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:33969
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474496157106-28370-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474496157106-28370-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.180.190:44391
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab
--principal=kudu/127.27.180.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.190:44391
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.180.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4955
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474496157106-28370-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:59:40.176666 4955 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:59:40.177649 4955 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:59:40.185947 4963 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:59:40.185976 4955 server_base.cc:1034] running on GCE node
W20260811 18:59:40.185942 4961 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:40.185942 4970 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:59:40.186712 4955 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:59:40.187703 4955 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:59:40.188845 4955 hybrid_clock.cc:648] HybridClock initialized: now 1786474780188833 us; error 34 us; skew 500 ppm
Aug 11 18:59:40 dist-test-slave-2j7r krb5kdc[4925](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474780, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:59:40.195595 4955 init.cc:377] Logged in from keytab as kudu/127.27.180.190@KRBTEST.COM (short username kudu)
I20260811 18:59:40.196456 4955 webserver.cc:460] Webserver started at http://127.27.180.190:43851/ using document root <none> and password file <none>
I20260811 18:59:40.196970 4955 fs_manager.cc:359] Metadata directory not provided
I20260811 18:59:40.197036 4955 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:59:40.197268 4955 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:59:40.199249 4955 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474496157106-28370-0/minicluster-data/master-0/data/instance:
uuid: "3d983e8d4477442a95dc5882a7a59fd2"
format_stamp: "Formatted at 2026-08-11 18:59:40 on dist-test-slave-2j7r"
server_key: "ef7cf71106397ec3f2d154439fdcd419"
server_key_iv: "dd508f45451e8d7eeca1c467be8056a6"
server_key_version: "encryptionkey@0"
I20260811 18:59:40.199787 4955 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474496157106-28370-0/minicluster-data/master-0/wal/instance:
uuid: "3d983e8d4477442a95dc5882a7a59fd2"
format_stamp: "Formatted at 2026-08-11 18:59:40 on dist-test-slave-2j7r"
server_key: "ef7cf71106397ec3f2d154439fdcd419"
server_key_iv: "dd508f45451e8d7eeca1c467be8056a6"
server_key_version: "encryptionkey@0"
I20260811 18:59:40.203349 4955 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 18:59:40.205652 4982 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:40.206843 4955 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 18:59:40.206970 4955 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474496157106-28370-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474496157106-28370-0/minicluster-data/master-0/wal
uuid: "3d983e8d4477442a95dc5882a7a59fd2"
format_stamp: "Formatted at 2026-08-11 18:59:40 on dist-test-slave-2j7r"
server_key: "ef7cf71106397ec3f2d154439fdcd419"
server_key_iv: "dd508f45451e8d7eeca1c467be8056a6"
server_key_version: "encryptionkey@0"
I20260811 18:59:40.207064 4955 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474496157106-28370-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474496157106-28370-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474496157106-28370-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:59:40.228729 4955 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:59:40.231371 4955 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:59:40.231556 4955 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:59:40.238421 4955 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.190:44391
I20260811 18:59:40.238453 5109 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.190:44391 every 8 connection(s)
I20260811 18:59:40.239331 4955 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
I20260811 18:59:40.242131 5112 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:59:40.243968 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 4955
I20260811 18:59:40.244117 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786474496157106-28370-0/minicluster-data/master-0/wal/instance
I20260811 18:59:40.244422 28370 external_mini_cluster.cc:1428] Setting key c556dd3b2c1354e9d8fb7e69b5f6fe33
I20260811 18:59:40.248476 5112 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d983e8d4477442a95dc5882a7a59fd2: Bootstrap starting.
I20260811 18:59:40.251045 5112 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3d983e8d4477442a95dc5882a7a59fd2: Neither blocks nor log segments found. Creating new log.
I20260811 18:59:40.251784 5112 log.cc:824] T 00000000000000000000000000000000 P 3d983e8d4477442a95dc5882a7a59fd2: Log is configured to *not* fsync() on all Append() calls
I20260811 18:59:40.252894 5116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:40.245767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59071 (local address 127.27.180.190:44391)
0811 18:59:40.246181 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:40.246197 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:40.246232 (+ 35us) server_negotiation.cc:407] Connection header received
0811 18:59:40.246872 (+ 640us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:40.246890 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:40.247207 (+ 317us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:40.247469 (+ 262us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:40.247819 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:40.248824 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:40.250001 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:40.250446 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:40.250558 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:40.251324 (+ 766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:40.251329 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:40.251341 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:59:40.251713 (+ 372us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:40.251765 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:59:40.251788 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:40.251862 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:40.252010 (+ 148us) server_negotiation.cc:299] Negotiation successful
0811 18:59:40.252198 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":128,"threads_started":1}
I20260811 18:59:40.253700 5112 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d983e8d4477442a95dc5882a7a59fd2: No bootstrap required, opened a new log
I20260811 18:59:40.255661 28370 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:59:40.257954 5112 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3d983e8d4477442a95dc5882a7a59fd2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d983e8d4477442a95dc5882a7a59fd2" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 44391 } }
I20260811 18:59:40.258201 5112 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3d983e8d4477442a95dc5882a7a59fd2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:59:40.258272 5112 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3d983e8d4477442a95dc5882a7a59fd2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d983e8d4477442a95dc5882a7a59fd2, State: Initialized, Role: FOLLOWER
I20260811 18:59:40.258688 5112 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3d983e8d4477442a95dc5882a7a59fd2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d983e8d4477442a95dc5882a7a59fd2" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 44391 } }
I20260811 18:59:40.258821 5112 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3d983e8d4477442a95dc5882a7a59fd2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:59:40.258894 5112 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3d983e8d4477442a95dc5882a7a59fd2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:59:40.258989 5112 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3d983e8d4477442a95dc5882a7a59fd2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:59:40.259809 5112 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3d983e8d4477442a95dc5882a7a59fd2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d983e8d4477442a95dc5882a7a59fd2" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 44391 } }
I20260811 18:59:40.260129 5112 leader_election.cc:302] T 00000000000000000000000000000000 P 3d983e8d4477442a95dc5882a7a59fd2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3d983e8d4477442a95dc5882a7a59fd2; no voters:
I20260811 18:59:40.260406 5112 leader_election.cc:288] T 00000000000000000000000000000000 P 3d983e8d4477442a95dc5882a7a59fd2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:59:40.260499 5131 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3d983e8d4477442a95dc5882a7a59fd2 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:59:40.260671 5131 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3d983e8d4477442a95dc5882a7a59fd2 [term 1 LEADER]: Becoming Leader. State: Replica: 3d983e8d4477442a95dc5882a7a59fd2, State: Running, Role: LEADER
I20260811 18:59:40.261013 5131 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3d983e8d4477442a95dc5882a7a59fd2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d983e8d4477442a95dc5882a7a59fd2" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 44391 } }
I20260811 18:59:40.261696 5112 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3d983e8d4477442a95dc5882a7a59fd2 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:59:40.261838 5116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:40.257920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36113 (local address 127.27.180.190:44391)
0811 18:59:40.258036 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:40.258040 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:40.258213 (+ 173us) server_negotiation.cc:407] Connection header received
0811 18:59:40.258343 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:40.258347 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:40.258405 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:40.258469 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:40.258714 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:40.259316 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:40.260109 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:40.260430 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:40.260523 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:40.261407 (+ 884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:40.261411 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:40.261414 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:59:40.261426 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:40.261450 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:59:40.261460 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:40.261527 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:40.261606 (+ 79us) server_negotiation.cc:299] Negotiation successful
0811 18:59:40.261653 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 18:59:40.262219 5131 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d983e8d4477442a95dc5882a7a59fd2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3d983e8d4477442a95dc5882a7a59fd2. Latest consensus state: current_term: 1 leader_uuid: "3d983e8d4477442a95dc5882a7a59fd2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d983e8d4477442a95dc5882a7a59fd2" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 44391 } } }
I20260811 18:59:40.262339 5131 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d983e8d4477442a95dc5882a7a59fd2 [sys.catalog]: This master's current role is: LEADER
I20260811 18:59:40.262758 5133 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d983e8d4477442a95dc5882a7a59fd2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3d983e8d4477442a95dc5882a7a59fd2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d983e8d4477442a95dc5882a7a59fd2" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 44391 } } }
I20260811 18:59:40.263001 5133 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d983e8d4477442a95dc5882a7a59fd2 [sys.catalog]: This master's current role is: LEADER
I20260811 18:59:40.263319 5141 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:59:40.266433 5141 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:59:40.271966 5141 catalog_manager.cc:1347] Generated new cluster ID: 1b834e0d8a4e486fb6706d8ea7896b6f
I20260811 18:59:40.272032 5141 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:59:40.278489 5141 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:59:40.279629 5141 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:59:40.285990 5141 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3d983e8d4477442a95dc5882a7a59fd2: Generated new TSK 0
I20260811 18:59:40.286725 5141 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:59:40.292783 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 4955
2026-08-11T18:59:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1574 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474496157106-28370-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:59:40 dist-test-slave-2j7r krb5kdc[5167](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:59:40 dist-test-slave-2j7r krb5kdc[5167](info): set up 2 sockets
Aug 11 18:59:40 dist-test-slave-2j7r krb5kdc[5167](info): commencing operation
krb5kdc: starting...
W20260811 18:59:41.514057 28370 mini_kdc.cc:121] Time spent starting KDC: real 1.172s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474496157106-28370-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474496157106-28370-0/krb5kdc/test-user.keytab.
Aug 11 18:59:41 dist-test-slave-2j7r krb5kdc[5167](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474781, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:59:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:59:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.180.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.180.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.180.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab.
Entry for principal kudu/127.27.180.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab.
I20260811 18:59:41.630411 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474496157106-28370-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474496157106-28370-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474496157106-28370-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.190:46213
--webserver_interface=127.27.180.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab
--principal=kudu/127.27.180.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:38135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.180.190:46213
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:59:41.739379 5195 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:59:41.739614 5195 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:59:41.739653 5195 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:59:41.743539 5195 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:59:41.743588 5195 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:59:41.743606 5195 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:59:41.743621 5195 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:59:41.743640 5195 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:59:41.748764 5195 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:38135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474496157106-28370-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474496157106-28370-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.180.190:46213
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab
--principal=kudu/127.27.180.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.190:46213
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.180.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5195
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474496157106-28370-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:59:41.749732 5195 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:59:41.750674 5195 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:59:41.758783 5205 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:41.758884 5208 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:59:41.758916 5195 server_base.cc:1034] running on GCE node
W20260811 18:59:41.758783 5210 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:59:41.759608 5195 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:59:41.760638 5195 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:59:41.761780 5195 hybrid_clock.cc:648] HybridClock initialized: now 1786474781761762 us; error 34 us; skew 500 ppm
Aug 11 18:59:41 dist-test-slave-2j7r krb5kdc[5167](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474781, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:59:41.764381 5195 init.cc:377] Logged in from keytab as kudu/127.27.180.190@KRBTEST.COM (short username kudu)
I20260811 18:59:41.765271 5195 webserver.cc:460] Webserver started at http://127.27.180.190:41643/ using document root <none> and password file <none>
I20260811 18:59:41.765792 5195 fs_manager.cc:359] Metadata directory not provided
I20260811 18:59:41.765857 5195 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:59:41.766062 5195 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:59:41.768011 5195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474496157106-28370-0/minicluster-data/master-0/data/instance:
uuid: "e4a3be0e7f6f407bb06fea5ee42bbc81"
format_stamp: "Formatted at 2026-08-11 18:59:41 on dist-test-slave-2j7r"
server_key: "4d6209468672d0ecbbcc49ebadb1e87c"
server_key_iv: "572e53be4e90f237d9719c9acfbdebfc"
server_key_version: "encryptionkey@0"
I20260811 18:59:41.768543 5195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474496157106-28370-0/minicluster-data/master-0/wal/instance:
uuid: "e4a3be0e7f6f407bb06fea5ee42bbc81"
format_stamp: "Formatted at 2026-08-11 18:59:41 on dist-test-slave-2j7r"
server_key: "4d6209468672d0ecbbcc49ebadb1e87c"
server_key_iv: "572e53be4e90f237d9719c9acfbdebfc"
server_key_version: "encryptionkey@0"
I20260811 18:59:41.772051 5195 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.006s
I20260811 18:59:41.774283 5226 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:41.775410 5195 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 18:59:41.775518 5195 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474496157106-28370-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474496157106-28370-0/minicluster-data/master-0/wal
uuid: "e4a3be0e7f6f407bb06fea5ee42bbc81"
format_stamp: "Formatted at 2026-08-11 18:59:41 on dist-test-slave-2j7r"
server_key: "4d6209468672d0ecbbcc49ebadb1e87c"
server_key_iv: "572e53be4e90f237d9719c9acfbdebfc"
server_key_version: "encryptionkey@0"
I20260811 18:59:41.775604 5195 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474496157106-28370-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474496157106-28370-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474496157106-28370-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:59:41.779940 5195 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:59:41.782753 5195 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:59:41.782928 5195 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:59:41.790017 5195 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.190:46213
I20260811 18:59:41.790055 5340 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.190:46213 every 8 connection(s)
I20260811 18:59:41.790932 5195 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
I20260811 18:59:41.793586 5341 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:59:41.795964 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 5195
I20260811 18:59:41.796089 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786474496157106-28370-0/minicluster-data/master-0/wal/instance
I20260811 18:59:41.796331 28370 external_mini_cluster.cc:1428] Setting key 6748236cac58fac691e663c1879bc256
I20260811 18:59:41.799410 5341 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4a3be0e7f6f407bb06fea5ee42bbc81: Bootstrap starting.
I20260811 18:59:41.801568 5350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:41.797657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35619 (local address 127.27.180.190:46213)
0811 18:59:41.798078 (+ 421us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:41.798095 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:41.798138 (+ 43us) server_negotiation.cc:407] Connection header received
0811 18:59:41.798912 (+ 774us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:41.798933 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:41.799265 (+ 332us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:41.799627 (+ 362us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:41.800202 (+ 575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:41.800207 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:41.800220 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:59:41.800230 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:41.800284 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:59:41.800322 (+ 38us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:41.800456 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:41.800526 (+ 70us) server_negotiation.cc:299] Negotiation successful
0811 18:59:41.800675 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":260,"thread_start_us":93,"threads_started":1}
I20260811 18:59:41.802053 5341 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e4a3be0e7f6f407bb06fea5ee42bbc81: Neither blocks nor log segments found. Creating new log.
I20260811 18:59:41.802858 5341 log.cc:824] T 00000000000000000000000000000000 P e4a3be0e7f6f407bb06fea5ee42bbc81: Log is configured to *not* fsync() on all Append() calls
I20260811 18:59:41.804342 28370 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:59:41.805135 5341 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4a3be0e7f6f407bb06fea5ee42bbc81: No bootstrap required, opened a new log
I20260811 18:59:41.808485 5341 raft_consensus.cc:348] T 00000000000000000000000000000000 P e4a3be0e7f6f407bb06fea5ee42bbc81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4a3be0e7f6f407bb06fea5ee42bbc81" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 46213 } }
I20260811 18:59:41.808681 5341 raft_consensus.cc:374] T 00000000000000000000000000000000 P e4a3be0e7f6f407bb06fea5ee42bbc81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 18:59:41.808738 5341 raft_consensus.cc:729] T 00000000000000000000000000000000 P e4a3be0e7f6f407bb06fea5ee42bbc81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4a3be0e7f6f407bb06fea5ee42bbc81, State: Initialized, Role: FOLLOWER
I20260811 18:59:41.809199 5341 consensus_queue.cc:260] T 00000000000000000000000000000000 P e4a3be0e7f6f407bb06fea5ee42bbc81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4a3be0e7f6f407bb06fea5ee42bbc81" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 46213 } }
I20260811 18:59:41.809370 5341 raft_consensus.cc:388] T 00000000000000000000000000000000 P e4a3be0e7f6f407bb06fea5ee42bbc81 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 18:59:41.809444 5341 raft_consensus.cc:482] T 00000000000000000000000000000000 P e4a3be0e7f6f407bb06fea5ee42bbc81 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 18:59:41.809544 5341 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e4a3be0e7f6f407bb06fea5ee42bbc81 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:59:41.810528 5350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:41.809030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56043 (local address 127.27.180.190:46213)
0811 18:59:41.809165 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:41.809169 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:41.809323 (+ 154us) server_negotiation.cc:407] Connection header received
0811 18:59:41.809475 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:41.809478 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:41.809516 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:41.809581 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:41.810173 (+ 592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:41.810175 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:41.810177 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 18:59:41.810186 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:41.810210 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 18:59:41.810219 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:41.810291 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:41.810394 (+ 103us) server_negotiation.cc:299] Negotiation successful
0811 18:59:41.810425 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 18:59:41.810581 5341 raft_consensus.cc:504] T 00000000000000000000000000000000 P e4a3be0e7f6f407bb06fea5ee42bbc81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4a3be0e7f6f407bb06fea5ee42bbc81" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 46213 } }
I20260811 18:59:41.810945 5341 leader_election.cc:302] T 00000000000000000000000000000000 P e4a3be0e7f6f407bb06fea5ee42bbc81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e4a3be0e7f6f407bb06fea5ee42bbc81; no voters:
I20260811 18:59:41.811265 5341 leader_election.cc:288] T 00000000000000000000000000000000 P e4a3be0e7f6f407bb06fea5ee42bbc81 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 18:59:41.811350 5359 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e4a3be0e7f6f407bb06fea5ee42bbc81 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:59:41.811578 5359 raft_consensus.cc:686] T 00000000000000000000000000000000 P e4a3be0e7f6f407bb06fea5ee42bbc81 [term 1 LEADER]: Becoming Leader. State: Replica: e4a3be0e7f6f407bb06fea5ee42bbc81, State: Running, Role: LEADER
I20260811 18:59:41.811862 5359 consensus_queue.cc:237] T 00000000000000000000000000000000 P e4a3be0e7f6f407bb06fea5ee42bbc81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4a3be0e7f6f407bb06fea5ee42bbc81" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 46213 } }
I20260811 18:59:41.812356 5341 sys_catalog.cc:564] T 00000000000000000000000000000000 P e4a3be0e7f6f407bb06fea5ee42bbc81 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:59:41.812862 5359 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4a3be0e7f6f407bb06fea5ee42bbc81 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e4a3be0e7f6f407bb06fea5ee42bbc81. Latest consensus state: current_term: 1 leader_uuid: "e4a3be0e7f6f407bb06fea5ee42bbc81" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4a3be0e7f6f407bb06fea5ee42bbc81" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 46213 } } }
I20260811 18:59:41.813031 5359 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4a3be0e7f6f407bb06fea5ee42bbc81 [sys.catalog]: This master's current role is: LEADER
I20260811 18:59:41.813300 5360 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4a3be0e7f6f407bb06fea5ee42bbc81 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e4a3be0e7f6f407bb06fea5ee42bbc81" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4a3be0e7f6f407bb06fea5ee42bbc81" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 46213 } } }
I20260811 18:59:41.813436 5360 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4a3be0e7f6f407bb06fea5ee42bbc81 [sys.catalog]: This master's current role is: LEADER
I20260811 18:59:41.813709 5373 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:59:41.816633 5373 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:59:41.822077 5373 catalog_manager.cc:1347] Generated new cluster ID: 4b2ba1781e184d8d838075483525c8f9
I20260811 18:59:41.822136 5373 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:59:41.830091 5373 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:59:41.830431 5373 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:59:41.841545 5373 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e4a3be0e7f6f407bb06fea5ee42bbc81: Generated new TSK 0
I20260811 18:59:41.842250 5373 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 18:59:41.850426 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 5195
2026-08-11T18:59:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1532 ms)
[----------] 18 tests from AuthTokenIssuingTest (28170 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474496157106-28370-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:59:41 dist-test-slave-2j7r krb5kdc[5404](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:59:41 dist-test-slave-2j7r krb5kdc[5404](info): set up 2 sockets
Aug 11 18:59:41 dist-test-slave-2j7r krb5kdc[5404](info): commencing operation
krb5kdc: starting...
W20260811 18:59:43.033504 28370 mini_kdc.cc:121] Time spent starting KDC: real 1.160s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474496157106-28370-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474496157106-28370-0/krb5kdc/test-user.keytab.
Aug 11 18:59:43 dist-test-slave-2j7r krb5kdc[5404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474783, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:59:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:59:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.180.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.180.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.180.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab.
Entry for principal kudu/127.27.180.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab.
I20260811 18:59:43.151537 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474496157106-28370-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474496157106-28370-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474496157106-28370-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.190:36235
--webserver_interface=127.27.180.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab
--principal=kudu/127.27.180.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:36891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.180.190:36235,127.27.180.189:36587,127.27.180.188:41433
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:59:43.258117 5433 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 18:59:43.258371 5433 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:59:43.258392 5433 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:59:43.258427 5433 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:59:43.262287 5433 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:59:43.262328 5433 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:59:43.262344 5433 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:59:43.262365 5433 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:59:43.262379 5433 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:59:43.267336 5433 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:36891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474496157106-28370-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474496157106-28370-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.180.190:36235,127.27.180.189:36587,127.27.180.188:41433
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab
--principal=kudu/127.27.180.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.190:36235
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.180.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5433
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474496157106-28370-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:59:43.268234 5433 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:59:43.269145 5433 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:59:43.276898 5441 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:43.276875 5446 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:43.276887 5440 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:59:43.277209 5433 server_base.cc:1034] running on GCE node
I20260811 18:59:43.277725 5433 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:59:43.278721 5433 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:59:43.279886 5433 hybrid_clock.cc:648] HybridClock initialized: now 1786474783279863 us; error 28 us; skew 500 ppm
Aug 11 18:59:43 dist-test-slave-2j7r krb5kdc[5404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474783, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:59:43.282438 5433 init.cc:377] Logged in from keytab as kudu/127.27.180.190@KRBTEST.COM (short username kudu)
I20260811 18:59:43.283233 5433 webserver.cc:460] Webserver started at http://127.27.180.190:43079/ using document root <none> and password file <none>
I20260811 18:59:43.283738 5433 fs_manager.cc:359] Metadata directory not provided
I20260811 18:59:43.283802 5433 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:59:43.284005 5433 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:59:43.285982 5433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474496157106-28370-0/minicluster-data/master-0/data/instance:
uuid: "2de63f29c4a143e19831d8c341185365"
format_stamp: "Formatted at 2026-08-11 18:59:43 on dist-test-slave-2j7r"
server_key: "596acc9a643f63880642ec87d5b0ea45"
server_key_iv: "758c53879ddfdbb34db1294986160b7f"
server_key_version: "encryptionkey@0"
I20260811 18:59:43.286521 5433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474496157106-28370-0/minicluster-data/master-0/wal/instance:
uuid: "2de63f29c4a143e19831d8c341185365"
format_stamp: "Formatted at 2026-08-11 18:59:43 on dist-test-slave-2j7r"
server_key: "596acc9a643f63880642ec87d5b0ea45"
server_key_iv: "758c53879ddfdbb34db1294986160b7f"
server_key_version: "encryptionkey@0"
I20260811 18:59:43.289952 5433 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 18:59:43.292294 5460 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:43.293464 5433 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 18:59:43.293574 5433 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474496157106-28370-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474496157106-28370-0/minicluster-data/master-0/wal
uuid: "2de63f29c4a143e19831d8c341185365"
format_stamp: "Formatted at 2026-08-11 18:59:43 on dist-test-slave-2j7r"
server_key: "596acc9a643f63880642ec87d5b0ea45"
server_key_iv: "758c53879ddfdbb34db1294986160b7f"
server_key_version: "encryptionkey@0"
I20260811 18:59:43.293661 5433 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474496157106-28370-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474496157106-28370-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474496157106-28370-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:59:43.319845 5433 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:59:43.322515 5433 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:59:43.322687 5433 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:59:43.329321 5433 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.190:36235
I20260811 18:59:43.329584 5553 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.190:36235 every 8 connection(s)
I20260811 18:59:43.330222 5433 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
I20260811 18:59:43.332811 5555 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:59:43.336344 5555 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.180.190" port: 36235 } has no permanent_uuid. Determining permanent_uuid...
I20260811 18:59:43.336963 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 5433
I20260811 18:59:43.337100 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474496157106-28370-0/minicluster-data/master-0/wal/instance
I20260811 18:59:43.337350 28370 external_mini_cluster.cc:1428] Setting key 7340e6b04e1549a22c68c6adff9ac06f
Aug 11 18:59:43 dist-test-slave-2j7r krb5kdc[5404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474783, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.190@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.27.180.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.180.189@KRBTEST.COM" created.
I20260811 18:59:43.350836 5557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:43.338572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54596 (local address 127.27.180.190:36235)
0811 18:59:43.339001 (+ 429us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:43.339024 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:43.339243 (+ 219us) server_negotiation.cc:407] Connection header received
0811 18:59:43.340436 (+ 1193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:43.340441 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:43.340612 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:43.340757 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:43.341572 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:43.342883 (+ 1311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:43.343805 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:43.344384 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:43.344550 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:43.346655 (+ 2105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:43.346660 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:43.346670 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:59:43.346691 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:43.347972 (+ 1281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:43.348380 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:43.348385 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:43.348386 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:43.348482 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:43.348725 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:43.348727 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:43.348728 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:43.349167 (+ 439us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:43.349296 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:43.349634 (+ 338us) server_negotiation.cc:299] Negotiation successful
0811 18:59:43.349843 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":48,"server-negotiator.queue_time_us":295,"thread_start_us":111,"threads_started":1}
I20260811 18:59:43.350836 5559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:43.338573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:36235 (local address 127.0.0.1:54596)
0811 18:59:43.339107 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:43.339142 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0811 18:59:43.340133 (+ 991us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:59:43.340800 (+ 667us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:59:43.340805 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:59:43.341175 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:59:43.341426 (+ 251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:43.341440 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:43.342989 (+ 1549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:43.342992 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:43.343688 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:43.343697 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:43.344511 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:43.344513 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:43.344589 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:43.345125 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:59:43.345145 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:59:43.346523 (+ 1378us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:59:43.348066 (+ 1543us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:59:43.348074 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:59:43.348081 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:59:43.348288 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:59:43.348556 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:59:43.348559 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:59:43.348560 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:59:43.348662 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:59:43.349266 (+ 604us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:59:43.349271 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:59:43.349405 (+ 134us) client_negotiation.cc:769] Sending connection context
0811 18:59:43.349552 (+ 147us) client_negotiation.cc:240] Negotiation successful
0811 18:59:43.349903 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":377,"mutex_wait_us":102,"spinlock_wait_cycles":24320,"thread_start_us":136,"threads_started":1}
I20260811 18:59:43.353022 5555 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.180.189" port: 36587 } has no permanent_uuid. Determining permanent_uuid...
I20260811 18:59:43.353735 5559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:43.353441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.189:36587 (local address 127.0.0.1:43852)
0811 18:59:43.353571 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:43.353657 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.189:36587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 18:59:43.353938 5462 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.27.180.189:36587: connect: Connection refused (error 111)
I20260811 18:59:43.354725 5559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:43.354494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.189:36587 (local address 127.0.0.1:43858)
0811 18:59:43.354606 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:43.354661 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.189:36587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260811 18:59:43.354875 5555 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.180.189:36587: Network error: Client connection negotiation failed: client connection to 127.27.180.189:36587: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.180.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.189.keytab.
Entry for principal kudu/127.27.180.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.189.keytab.
I20260811 18:59:43.363142 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474496157106-28370-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474496157106-28370-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474496157106-28370-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474496157106-28370-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.189:36587
--webserver_interface=127.27.180.189
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.189.keytab
--principal=kudu/127.27.180.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:36891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.180.190:36235,127.27.180.189:36587,127.27.180.188:41433
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 18:59:43.404088 5555 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.180.189" port: 36587 } attempt: 1
I20260811 18:59:43.404954 5559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:43.404641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.189:36587 (local address 127.0.0.1:43872)
0811 18:59:43.404799 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:43.404860 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.189:36587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260811 18:59:43.405660 5559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:43.405463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.189:36587 (local address 127.0.0.1:43886)
0811 18:59:43.405554 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:43.405598 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.189:36587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260811 18:59:43.405776 5555 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.180.189:36587: Network error: Client connection negotiation failed: client connection to 127.27.180.189:36587: connect: Connection refused (error 111)
W20260811 18:59:43.473675 5571 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 18:59:43.473961 5571 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:59:43.473994 5571 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
I20260811 18:59:43.473919 5555 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.180.189" port: 36587 } attempt: 2
W20260811 18:59:43.474061 5571 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260811 18:59:43.474730 5559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:43.474450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.189:36587 (local address 127.0.0.1:43898)
0811 18:59:43.474595 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:43.474653 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.189:36587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260811 18:59:43.475389 5559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:43.475202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.189:36587 (local address 127.0.0.1:43906)
0811 18:59:43.475287 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:43.475329 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.189:36587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260811 18:59:43.475499 5555 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.180.189:36587: Network error: Client connection negotiation failed: client connection to 127.27.180.189:36587: connect: Connection refused (error 111)
W20260811 18:59:43.478528 5571 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:59:43.478586 5571 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:59:43.478612 5571 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:59:43.478648 5571 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:59:43.478684 5571 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:59:43.484000 5571 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:36891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474496157106-28370-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474496157106-28370-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.27.180.190:36235,127.27.180.189:36587,127.27.180.188:41433
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.189.keytab
--principal=kudu/127.27.180.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.189:36587
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474496157106-28370-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.27.180.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5571
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474496157106-28370-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:59:43.485146 5571 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:59:43.486253 5571 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:59:43.494865 5589 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:43.494835 5590 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:43.494839 5592 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:59:43.495115 5571 server_base.cc:1034] running on GCE node
I20260811 18:59:43.495647 5571 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:59:43.496743 5571 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:59:43.497910 5571 hybrid_clock.cc:648] HybridClock initialized: now 1786474783497891 us; error 33 us; skew 500 ppm
Aug 11 18:59:43 dist-test-slave-2j7r krb5kdc[5404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474783, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:59:43.500700 5571 init.cc:377] Logged in from keytab as kudu/127.27.180.189@KRBTEST.COM (short username kudu)
I20260811 18:59:43.501582 5571 webserver.cc:460] Webserver started at http://127.27.180.189:40369/ using document root <none> and password file <none>
I20260811 18:59:43.502094 5571 fs_manager.cc:359] Metadata directory not provided
I20260811 18:59:43.502152 5571 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:59:43.502357 5571 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:59:43.504309 5571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474496157106-28370-0/minicluster-data/master-1/data/instance:
uuid: "e3c923fd30d944779227d74276d95d4f"
format_stamp: "Formatted at 2026-08-11 18:59:43 on dist-test-slave-2j7r"
server_key: "1686728232ef03a5b8d2450d55244a44"
server_key_iv: "05db9c6dcc88342eb11e9491e1de965b"
server_key_version: "encryptionkey@0"
I20260811 18:59:43.504830 5571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474496157106-28370-0/minicluster-data/master-1/wal/instance:
uuid: "e3c923fd30d944779227d74276d95d4f"
format_stamp: "Formatted at 2026-08-11 18:59:43 on dist-test-slave-2j7r"
server_key: "1686728232ef03a5b8d2450d55244a44"
server_key_iv: "05db9c6dcc88342eb11e9491e1de965b"
server_key_version: "encryptionkey@0"
I20260811 18:59:43.508260 5571 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260811 18:59:43.510527 5613 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:43.511780 5571 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 18:59:43.511878 5571 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474496157106-28370-0/minicluster-data/master-1/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474496157106-28370-0/minicluster-data/master-1/wal
uuid: "e3c923fd30d944779227d74276d95d4f"
format_stamp: "Formatted at 2026-08-11 18:59:43 on dist-test-slave-2j7r"
server_key: "1686728232ef03a5b8d2450d55244a44"
server_key_iv: "05db9c6dcc88342eb11e9491e1de965b"
server_key_version: "encryptionkey@0"
I20260811 18:59:43.511974 5571 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474496157106-28370-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474496157106-28370-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474496157106-28370-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:59:43.538254 5571 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:59:43.541004 5571 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:59:43.541193 5571 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:59:43.548180 5571 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.189:36587
I20260811 18:59:43.548452 5730 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.189:36587 every 8 connection(s)
I20260811 18:59:43.549158 5571 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474496157106-28370-0/minicluster-data/master-1/data/info.pb
I20260811 18:59:43.551874 5731 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:59:43.555523 5731 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.180.190" port: 36235 } has no permanent_uuid. Determining permanent_uuid...
I20260811 18:59:43.559172 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 5571
I20260811 18:59:43.559288 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474496157106-28370-0/minicluster-data/master-1/wal/instance
I20260811 18:59:43.559552 28370 external_mini_cluster.cc:1428] Setting key 3cac58a818c5298f92f86f277f0e606e
Aug 11 18:59:43 dist-test-slave-2j7r krb5kdc[5404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474783, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.189@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM
I20260811 18:59:43.566661 5555 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.180.189" port: 36587 } attempt: 3
WARNING: no policy specified for kudu/127.27.180.188@KRBTEST.COM; defaulting to no policy
I20260811 18:59:43.568373 5557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:43.557298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54610 (local address 127.27.180.190:36235)
0811 18:59:43.557433 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:43.557437 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:43.558251 (+ 814us) server_negotiation.cc:407] Connection header received
0811 18:59:43.559136 (+ 885us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:43.559140 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:43.559209 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:43.559280 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:43.560511 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:43.561040 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:43.562140 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:43.562383 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:43.562473 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:43.564772 (+ 2299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:43.564775 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:43.564777 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:59:43.564803 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:43.566157 (+ 1354us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:43.566645 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:43.566648 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:43.566649 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:43.566753 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:43.567142 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:43.567145 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:43.567147 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:43.567371 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:43.567440 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:43.568111 (+ 671us) server_negotiation.cc:299] Negotiation successful
0811 18:59:43.568218 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 18:59:43.569247 5733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:43.557636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:36235 (local address 127.0.0.1:54610)
0811 18:59:43.558060 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:43.558103 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 18:59:43.558937 (+ 834us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:59:43.559450 (+ 513us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:59:43.559457 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:59:43.559893 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:59:43.560363 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:43.560376 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:43.561169 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:43.561172 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:43.562015 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:43.562025 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:43.562528 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:43.562532 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:43.562691 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:43.563219 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:59:43.563235 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:59:43.564614 (+ 1379us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:59:43.566293 (+ 1679us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:59:43.566299 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:59:43.566313 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:59:43.566553 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:59:43.566856 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:59:43.566860 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:59:43.566862 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:59:43.567038 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:59:43.567522 (+ 484us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:59:43.567528 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:59:43.567816 (+ 288us) client_negotiation.cc:769] Sending connection context
0811 18:59:43.568034 (+ 218us) client_negotiation.cc:240] Negotiation successful
0811 18:59:43.568300 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":109,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.180.188@KRBTEST.COM" created.
I20260811 18:59:43.570712 5731 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.180.189" port: 36587 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 18:59:43 dist-test-slave-2j7r krb5kdc[5404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474783, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.190@KRBTEST.COM for kudu/127.27.180.189@KRBTEST.COM
Aug 11 18:59:43 dist-test-slave-2j7r krb5kdc[5404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474783, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.189@KRBTEST.COM for kudu/127.27.180.189@KRBTEST.COM
I20260811 18:59:43.577955 5559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:43.567105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.189:36587 (local address 127.0.0.1:43916)
0811 18:59:43.567306 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:43.567320 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 18:59:43.567384 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:59:43.568005 (+ 621us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:59:43.568007 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:59:43.568165 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:59:43.568695 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:43.568705 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:43.569897 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:43.569900 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:43.570654 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:43.570665 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:43.571530 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:43.571534 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:43.571638 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:43.572107 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:59:43.572120 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:59:43.573610 (+ 1490us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:59:43.576053 (+ 2443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:59:43.576056 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:59:43.576057 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:59:43.576236 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:59:43.576770 (+ 534us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:59:43.576773 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:59:43.576776 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:59:43.576873 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:59:43.577458 (+ 585us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:59:43.577462 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:59:43.577564 (+ 102us) client_negotiation.cc:769] Sending connection context
0811 18:59:43.577665 (+ 101us) client_negotiation.cc:240] Negotiation successful
0811 18:59:43.577797 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133}
I20260811 18:59:43.578089 5741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:43.567203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43916 (local address 127.27.180.189:36587)
0811 18:59:43.567602 (+ 399us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:43.567613 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:43.567634 (+ 21us) server_negotiation.cc:407] Connection header received
0811 18:59:43.567667 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:43.567672 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:43.567864 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:43.567998 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:43.568816 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:43.569801 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:43.570788 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:43.571356 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:43.571526 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:43.573741 (+ 2215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:43.573753 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:43.573766 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:59:43.573794 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:43.575946 (+ 2152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:43.576344 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:43.576351 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:43.576352 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:43.576469 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:43.576977 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:43.576982 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:43.576983 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:43.577340 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:43.577495 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:43.577792 (+ 297us) server_negotiation.cc:299] Negotiation successful
0811 18:59:43.577932 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":137,"threads_started":1}
I20260811 18:59:43.580170 5733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:43.571163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.189:36587 (local address 127.0.0.1:43924)
0811 18:59:43.571246 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:43.571258 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 18:59:43.571327 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:59:43.571791 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:59:43.571794 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:59:43.571965 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:59:43.572093 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:43.572100 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:43.572990 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:43.572993 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:43.573626 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:43.573637 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:43.574236 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:43.574241 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:43.574345 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:43.574737 (+ 392us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:59:43.574750 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:59:43.576198 (+ 1448us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:59:43.578937 (+ 2739us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:59:43.578940 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:59:43.578943 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:59:43.579114 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:59:43.579350 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:59:43.579353 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:59:43.579355 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:59:43.579431 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:59:43.579738 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:59:43.579741 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:59:43.579821 (+ 80us) client_negotiation.cc:769] Sending connection context
0811 18:59:43.579917 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 18:59:43.580033 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260811 18:59:43.580159 5555 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.180.188" port: 41433 } has no permanent_uuid. Determining permanent_uuid...
I20260811 18:59:43.580358 5742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:43.571279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43924 (local address 127.27.180.189:36587)
0811 18:59:43.571577 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:43.571586 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:43.571599 (+ 13us) server_negotiation.cc:407] Connection header received
0811 18:59:43.571628 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:43.571630 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:43.571669 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:43.571755 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:43.572241 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:43.572894 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:43.573787 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:43.574113 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:43.574213 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:43.576325 (+ 2112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:43.576329 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:43.576332 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:59:43.576355 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:43.578811 (+ 2456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:43.579196 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:43.579199 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:43.579201 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:43.579273 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:43.579505 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:43.579508 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:43.579510 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:43.579657 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:43.579737 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:43.580007 (+ 270us) server_negotiation.cc:299] Negotiation successful
0811 18:59:43.580131 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":227,"thread_start_us":113,"threads_started":1}
I20260811 18:59:43.580864 5559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:43.580584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.188:41433 (local address 127.0.0.1:34548)
0811 18:59:43.580752 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:43.580803 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.188:41433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
I20260811 18:59:43.581286 5731 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.180.188" port: 41433 } has no permanent_uuid. Determining permanent_uuid...
I20260811 18:59:43.581624 5559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:43.581333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.188:41433 (local address 127.0.0.1:34554)
0811 18:59:43.581501 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:43.581559 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.188:41433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
W20260811 18:59:43.581835 5555 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.180.188:41433: Network error: Client connection negotiation failed: client connection to 127.27.180.188:41433: connect: Connection refused (error 111)
I20260811 18:59:43.581909 5733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:43.581680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.188:41433 (local address 127.0.0.1:34570)
0811 18:59:43.581774 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:43.581849 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.188:41433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260811 18:59:43.582095 5619 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.27.180.188:41433: connect: Connection refused (error 111)
I20260811 18:59:43.582877 5733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:43.582676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.188:41433 (local address 127.0.0.1:34586)
0811 18:59:43.582779 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:43.582828 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.188:41433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260811 18:59:43.583006 5731 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.180.188:41433: Network error: Client connection negotiation failed: client connection to 127.27.180.188:41433: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.180.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.188.keytab.
Entry for principal kudu/127.27.180.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.188.keytab.
I20260811 18:59:43.587960 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474496157106-28370-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474496157106-28370-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474496157106-28370-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474496157106-28370-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.188:41433
--webserver_interface=127.27.180.188
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.188.keytab
--principal=kudu/127.27.180.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:36891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.180.190:36235,127.27.180.189:36587,127.27.180.188:41433
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 18:59:43.613000 5555 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.180.188" port: 41433 } attempt: 1
I20260811 18:59:43.613878 5559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:43.613560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.188:41433 (local address 127.0.0.1:34600)
0811 18:59:43.613721 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:43.613788 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.188:41433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260811 18:59:43.614617 5559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:43.614386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.188:41433 (local address 127.0.0.1:34610)
0811 18:59:43.614505 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:43.614555 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.188:41433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260811 18:59:43.614786 5555 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.180.188:41433: Network error: Client connection negotiation failed: client connection to 127.27.180.188:41433: connect: Connection refused (error 111)
I20260811 18:59:43.632264 5731 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.180.188" port: 41433 } attempt: 1
I20260811 18:59:43.633061 5733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:43.632763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.188:41433 (local address 127.0.0.1:34616)
0811 18:59:43.632901 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:43.632961 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.188:41433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260811 18:59:43.633813 5733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:43.633594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.188:41433 (local address 127.0.0.1:34620)
0811 18:59:43.633710 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:43.633757 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.188:41433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260811 18:59:43.633929 5731 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.180.188:41433: Network error: Client connection negotiation failed: client connection to 127.27.180.188:41433: connect: Connection refused (error 111)
I20260811 18:59:43.689940 5555 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.180.188" port: 41433 } attempt: 2
I20260811 18:59:43.690843 5559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:43.690520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.188:41433 (local address 127.0.0.1:34632)
0811 18:59:43.690683 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:43.690749 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.188:41433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260811 18:59:43.691672 5559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:43.691457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.188:41433 (local address 127.0.0.1:34640)
0811 18:59:43.691565 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:43.691611 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.188:41433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260811 18:59:43.691800 5555 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.180.188:41433: Network error: Client connection negotiation failed: client connection to 127.27.180.188:41433: connect: Connection refused (error 111)
W20260811 18:59:43.699213 5746 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 18:59:43.699465 5746 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:59:43.699512 5746 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:59:43.699580 5746 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260811 18:59:43.702075 5731 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.180.188" port: 41433 } attempt: 2
I20260811 18:59:43.702811 5733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:43.702565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.188:41433 (local address 127.0.0.1:34652)
0811 18:59:43.702689 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:43.702742 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.188:41433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260811 18:59:43.703496 5733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:43.703287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.188:41433 (local address 127.0.0.1:34654)
0811 18:59:43.703391 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:43.703439 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.188:41433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260811 18:59:43.703615 5731 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.180.188:41433: Network error: Client connection negotiation failed: client connection to 127.27.180.188:41433: connect: Connection refused (error 111)
W20260811 18:59:43.703871 5746 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:59:43.703922 5746 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:59:43.703949 5746 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:59:43.703979 5746 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:59:43.704005 5746 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:59:43.709158 5746 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:36891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474496157106-28370-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474496157106-28370-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.27.180.190:36235,127.27.180.189:36587,127.27.180.188:41433
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.188.keytab
--principal=kudu/127.27.180.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.188:41433
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474496157106-28370-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.27.180.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5746
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474496157106-28370-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:59:43.710096 5746 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:59:43.711023 5746 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 18:59:43.719086 5746 server_base.cc:1034] running on GCE node
W20260811 18:59:43.718993 5755 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:43.718999 5756 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:43.718994 5758 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:59:43.719861 5746 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:59:43.720856 5746 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:59:43.722010 5746 hybrid_clock.cc:648] HybridClock initialized: now 1786474783721982 us; error 36 us; skew 500 ppm
Aug 11 18:59:43 dist-test-slave-2j7r krb5kdc[5404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474783, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:59:43.724561 5746 init.cc:377] Logged in from keytab as kudu/127.27.180.188@KRBTEST.COM (short username kudu)
I20260811 18:59:43.725405 5746 webserver.cc:460] Webserver started at http://127.27.180.188:43167/ using document root <none> and password file <none>
I20260811 18:59:43.725890 5746 fs_manager.cc:359] Metadata directory not provided
I20260811 18:59:43.725957 5746 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:59:43.726119 5746 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:59:43.727989 5746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474496157106-28370-0/minicluster-data/master-2/data/instance:
uuid: "b3fdaa1655204114938dc2ee6b906d99"
format_stamp: "Formatted at 2026-08-11 18:59:43 on dist-test-slave-2j7r"
server_key: "81b54a0329e1802ad56a3ddd07e0960b"
server_key_iv: "9e591d9af083fce73d9cff72b2984a92"
server_key_version: "encryptionkey@0"
I20260811 18:59:43.728478 5746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474496157106-28370-0/minicluster-data/master-2/wal/instance:
uuid: "b3fdaa1655204114938dc2ee6b906d99"
format_stamp: "Formatted at 2026-08-11 18:59:43 on dist-test-slave-2j7r"
server_key: "81b54a0329e1802ad56a3ddd07e0960b"
server_key_iv: "9e591d9af083fce73d9cff72b2984a92"
server_key_version: "encryptionkey@0"
I20260811 18:59:43.732043 5746 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260811 18:59:43.734309 5772 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:43.735396 5746 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 18:59:43.735491 5746 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474496157106-28370-0/minicluster-data/master-2/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474496157106-28370-0/minicluster-data/master-2/wal
uuid: "b3fdaa1655204114938dc2ee6b906d99"
format_stamp: "Formatted at 2026-08-11 18:59:43 on dist-test-slave-2j7r"
server_key: "81b54a0329e1802ad56a3ddd07e0960b"
server_key_iv: "9e591d9af083fce73d9cff72b2984a92"
server_key_version: "encryptionkey@0"
I20260811 18:59:43.735569 5746 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474496157106-28370-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474496157106-28370-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474496157106-28370-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:59:43.753896 5746 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:59:43.756804 5746 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:59:43.756949 5746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:59:43.763414 5746 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.188:41433
I20260811 18:59:43.763689 5882 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.188:41433 every 8 connection(s)
I20260811 18:59:43.764317 5746 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474496157106-28370-0/minicluster-data/master-2/data/info.pb
I20260811 18:59:43.766979 5883 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:59:43.770620 5883 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.180.190" port: 36235 } has no permanent_uuid. Determining permanent_uuid...
I20260811 18:59:43.774160 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 5746
I20260811 18:59:43.774309 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786474496157106-28370-0/minicluster-data/master-2/wal/instance
I20260811 18:59:43.774626 28370 external_mini_cluster.cc:1428] Setting key ab9f602903cbaa00ff4017f72dcabc21
Aug 11 18:59:43 dist-test-slave-2j7r krb5kdc[5404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474783, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.188@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM
Aug 11 18:59:43 dist-test-slave-2j7r krb5kdc[5404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474783, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.180.188@KRBTEST.COM
I20260811 18:59:43.784953 5557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:43.772408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54626 (local address 127.27.180.190:36235)
0811 18:59:43.772575 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:43.772580 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:43.773502 (+ 922us) server_negotiation.cc:407] Connection header received
0811 18:59:43.774619 (+ 1117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:43.774621 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:43.774673 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:43.774764 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:43.776221 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:43.776990 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:43.778558 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:43.778887 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:43.779009 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:43.781492 (+ 2483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:43.781495 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:43.781497 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:59:43.781523 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:43.782881 (+ 1358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:43.783447 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:43.783449 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:43.783451 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:43.783526 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:43.783863 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:43.783875 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:43.783877 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:43.784035 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:43.784102 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:43.784693 (+ 591us) server_negotiation.cc:299] Negotiation successful
0811 18:59:43.784816 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 18:59:43.785897 5885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:43.772724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:36235 (local address 127.0.0.1:54626)
0811 18:59:43.773347 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:43.773405 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 18:59:43.774390 (+ 985us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:59:43.774947 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:59:43.774955 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:59:43.775421 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:59:43.776054 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:43.776070 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:43.777148 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:43.777151 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:43.778420 (+ 1269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:43.778431 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:43.779028 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:43.779031 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:43.779173 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:43.779745 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:59:43.779763 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:59:43.781344 (+ 1581us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:59:43.782981 (+ 1637us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:59:43.782992 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:59:43.783001 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:59:43.783344 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:59:43.783609 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:59:43.783612 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:59:43.783614 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:59:43.783780 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:59:43.784121 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:59:43.784127 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:59:43.784411 (+ 284us) client_negotiation.cc:769] Sending connection context
0811 18:59:43.784603 (+ 192us) client_negotiation.cc:240] Negotiation successful
0811 18:59:43.784876 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"mutex_wait_us":33,"thread_start_us":113,"threads_started":1}
I20260811 18:59:43.785897 5890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:43.775346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34664 (local address 127.27.180.188:41433)
0811 18:59:43.775736 (+ 390us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:43.775747 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:43.775771 (+ 24us) server_negotiation.cc:407] Connection header received
0811 18:59:43.775884 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:43.775890 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:43.776070 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:43.776225 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:43.777229 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:43.778486 (+ 1257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:43.779471 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:43.779892 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:43.779970 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:43.782470 (+ 2500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:43.782476 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:43.782486 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:59:43.782507 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:43.784099 (+ 1592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:43.784471 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:43.784477 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:43.784479 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:43.784558 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:43.784818 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:43.784821 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:43.784822 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:43.785075 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:43.785317 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:43.785475 (+ 158us) server_negotiation.cc:299] Negotiation successful
0811 18:59:43.785607 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":42,"server-negotiator.queue_time_us":298,"thread_start_us":151,"threads_started":1}
I20260811 18:59:43.787155 5883 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.180.189" port: 36587 } has no permanent_uuid. Determining permanent_uuid...
I20260811 18:59:43.790949 5555 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.180.188" port: 41433 } attempt: 3
Aug 11 18:59:43 dist-test-slave-2j7r krb5kdc[5404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474783, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.188@KRBTEST.COM for kudu/127.27.180.189@KRBTEST.COM
Aug 11 18:59:43 dist-test-slave-2j7r krb5kdc[5404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474783, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM
I20260811 18:59:43.794762 5731 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.180.188" port: 41433 } attempt: 3
Aug 11 18:59:43 dist-test-slave-2j7r krb5kdc[5404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474783, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.190@KRBTEST.COM for kudu/127.27.180.188@KRBTEST.COM
I20260811 18:59:43.797654 5557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:43.787995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54636 (local address 127.27.180.190:36235)
0811 18:59:43.788107 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:43.788111 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:43.788123 (+ 12us) server_negotiation.cc:407] Connection header received
0811 18:59:43.788202 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:43.788205 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:43.788257 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:43.788334 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:43.788919 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:43.789535 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:43.790302 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:43.790514 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:43.790604 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:43.793372 (+ 2768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:43.793375 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:43.793377 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:59:43.793398 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:43.795697 (+ 2299us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:43.796177 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:43.796180 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:43.796182 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:43.796275 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:43.796576 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:43.796579 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:43.796581 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:43.796783 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:43.796956 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:43.797397 (+ 441us) server_negotiation.cc:299] Negotiation successful
0811 18:59:43.797505 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 18:59:43.797912 5742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:43.787754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43934 (local address 127.27.180.189:36587)
0811 18:59:43.787868 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:43.787873 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:43.787892 (+ 19us) server_negotiation.cc:407] Connection header received
0811 18:59:43.787937 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:43.787940 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:43.787987 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:43.788067 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:43.788589 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:43.789294 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:43.790085 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:43.790415 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:43.790494 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:43.792571 (+ 2077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:43.792575 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:43.792578 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:59:43.792604 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:43.794985 (+ 2381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:43.795488 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:43.795492 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:43.795494 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:43.795579 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:43.795892 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:43.795896 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:43.795898 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:43.796252 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:43.796341 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:43.797581 (+ 1240us) server_negotiation.cc:299] Negotiation successful
0811 18:59:43.797729 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":125,"server-negotiator.queue_time_us":34}
I20260811 18:59:43.799031 5885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:43.787627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.189:36587 (local address 127.0.0.1:43934)
0811 18:59:43.787770 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:43.787782 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 18:59:43.787852 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:59:43.788191 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:59:43.788194 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:59:43.788360 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:59:43.788464 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:43.788469 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:43.789392 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:43.789395 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:43.789977 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:43.789986 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:43.790512 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:43.790515 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:43.790607 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:43.791028 (+ 421us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:59:43.791043 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:59:43.792413 (+ 1370us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:59:43.795116 (+ 2703us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:59:43.795119 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:59:43.795121 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:59:43.795353 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:59:43.795694 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:59:43.795697 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:59:43.795699 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:59:43.795787 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:59:43.797282 (+ 1495us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:59:43.797286 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:59:43.797379 (+ 93us) client_negotiation.cc:769] Sending connection context
0811 18:59:43.798748 (+ 1369us) client_negotiation.cc:240] Negotiation successful
0811 18:59:43.798884 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":86}
Aug 11 18:59:43 dist-test-slave-2j7r krb5kdc[5404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474783, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.189@KRBTEST.COM for kudu/127.27.180.188@KRBTEST.COM
I20260811 18:59:43.800187 5890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:43.791504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34670 (local address 127.27.180.188:41433)
0811 18:59:43.791602 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:43.791606 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:43.791678 (+ 72us) server_negotiation.cc:407] Connection header received
0811 18:59:43.791799 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:43.791802 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:43.791844 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:43.791924 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:43.792332 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:43.792904 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:43.793749 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:43.794016 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:43.794147 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:43.796118 (+ 1971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:43.796122 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:43.796124 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:59:43.796146 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:43.798311 (+ 2165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:43.799006 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:43.799009 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:43.799010 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:43.799089 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:43.799403 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:43.799406 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:43.799408 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:43.799601 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:43.799724 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:43.799971 (+ 247us) server_negotiation.cc:299] Negotiation successful
0811 18:59:43.800066 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 18:59:43.800186 5559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:43.791455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.188:41433 (local address 127.0.0.1:34670)
0811 18:59:43.791621 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:43.791634 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:59:43.791716 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:59:43.791957 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:59:43.791960 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:59:43.792126 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:59:43.792212 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:43.792219 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:43.793039 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:43.793043 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:43.793657 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:43.793667 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:43.794157 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:43.794160 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:43.794240 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:43.794556 (+ 316us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:59:43.794571 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:59:43.796008 (+ 1437us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:59:43.798440 (+ 2432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:59:43.798444 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:59:43.798445 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:59:43.798672 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:59:43.799199 (+ 527us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:59:43.799202 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:59:43.799204 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:59:43.799286 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:59:43.799720 (+ 434us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:59:43.799723 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:59:43.799811 (+ 88us) client_negotiation.cc:769] Sending connection context
0811 18:59:43.799916 (+ 105us) client_negotiation.cc:240] Negotiation successful
0811 18:59:43.800035 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100}
I20260811 18:59:43.800446 5883 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.180.188" port: 41433 } has no permanent_uuid. Determining permanent_uuid...
I20260811 18:59:43.803601 5555 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2de63f29c4a143e19831d8c341185365: Bootstrap starting.
Aug 11 18:59:43 dist-test-slave-2j7r krb5kdc[5404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474783, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.180.189@KRBTEST.COM
I20260811 18:59:43.804538 5733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:43.795129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.188:41433 (local address 127.0.0.1:34684)
0811 18:59:43.795426 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:43.795440 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 18:59:43.795514 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:59:43.795780 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:59:43.795783 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:59:43.795943 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:59:43.796063 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:43.796071 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:43.797369 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:43.797372 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:43.798090 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:43.798101 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:43.798529 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:43.798533 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:43.798622 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:43.798993 (+ 371us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:59:43.799007 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:59:43.801248 (+ 2241us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:59:43.803014 (+ 1766us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:59:43.803017 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:59:43.803019 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:59:43.803261 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:59:43.803552 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:59:43.803555 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:59:43.803557 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:59:43.803645 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:59:43.804062 (+ 417us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:59:43.804065 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:59:43.804160 (+ 95us) client_negotiation.cc:769] Sending connection context
0811 18:59:43.804267 (+ 107us) client_negotiation.cc:240] Negotiation successful
0811 18:59:43.804397 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223}
I20260811 18:59:43.804610 5891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:43.795162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34684 (local address 127.27.180.188:41433)
0811 18:59:43.795468 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:43.795476 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:43.795489 (+ 13us) server_negotiation.cc:407] Connection header received
0811 18:59:43.795593 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:43.795596 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:43.795635 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:43.796391 (+ 756us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:43.796477 (+ 86us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:43.797231 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:43.798214 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:43.798428 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:43.798508 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:43.801381 (+ 2873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:43.801385 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:43.801387 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:59:43.801410 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:43.802920 (+ 1510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:43.803358 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:43.803361 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:43.803363 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:43.803459 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:43.803753 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:43.803756 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:43.803758 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:43.803956 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:43.804054 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:43.804328 (+ 274us) server_negotiation.cc:299] Negotiation successful
0811 18:59:43.804433 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":251,"thread_start_us":111,"threads_started":1}
I20260811 18:59:43.806273 5555 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2de63f29c4a143e19831d8c341185365: Neither blocks nor log segments found. Creating new log.
I20260811 18:59:43.807152 5731 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3c923fd30d944779227d74276d95d4f: Bootstrap starting.
I20260811 18:59:43.807154 5555 log.cc:824] T 00000000000000000000000000000000 P 2de63f29c4a143e19831d8c341185365: Log is configured to *not* fsync() on all Append() calls
Aug 11 18:59:43 dist-test-slave-2j7r krb5kdc[5404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474783, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.188@KRBTEST.COM for kudu/127.27.180.188@KRBTEST.COM
I20260811 18:59:43.809329 5555 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2de63f29c4a143e19831d8c341185365: No bootstrap required, opened a new log
I20260811 18:59:43.809450 5731 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e3c923fd30d944779227d74276d95d4f: Neither blocks nor log segments found. Creating new log.
I20260811 18:59:43.810272 5731 log.cc:824] T 00000000000000000000000000000000 P e3c923fd30d944779227d74276d95d4f: Log is configured to *not* fsync() on all Append() calls
I20260811 18:59:43.812716 5731 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3c923fd30d944779227d74276d95d4f: No bootstrap required, opened a new log
I20260811 18:59:43.812852 5885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:43.800846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.188:41433 (local address 127.0.0.1:34696)
0811 18:59:43.800933 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:43.800945 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 18:59:43.801010 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:59:43.805328 (+ 4318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:59:43.805331 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:59:43.805487 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:59:43.805569 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:43.805575 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:43.806477 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:43.806481 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:43.807198 (+ 717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:43.807210 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:43.807835 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:43.807839 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:43.807917 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:43.808304 (+ 387us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:59:43.808317 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:59:43.809652 (+ 1335us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:59:43.811472 (+ 1820us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:59:43.811475 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:59:43.811477 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:59:43.811694 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:59:43.811942 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:59:43.811944 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:59:43.811946 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:59:43.812015 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:59:43.812385 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:59:43.812387 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:59:43.812480 (+ 93us) client_negotiation.cc:769] Sending connection context
0811 18:59:43.812592 (+ 112us) client_negotiation.cc:240] Negotiation successful
0811 18:59:43.812721 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260811 18:59:43.813227 5891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:43.800918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34696 (local address 127.27.180.188:41433)
0811 18:59:43.805122 (+ 4204us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:43.805126 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:43.805137 (+ 11us) server_negotiation.cc:407] Connection header received
0811 18:59:43.805165 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:43.805167 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:43.805213 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:43.805290 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:43.805688 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:43.806376 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:43.807396 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:43.807713 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:43.807813 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:43.809783 (+ 1970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:43.809787 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:43.809789 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:59:43.809813 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:43.811351 (+ 1538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:43.811789 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:43.811792 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:43.811794 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:43.811858 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:43.812100 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:43.812102 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:43.812104 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:43.812277 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:43.812386 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:43.812648 (+ 262us) server_negotiation.cc:299] Negotiation successful
0811 18:59:43.812850 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":4129}
I20260811 18:59:43.813336 5555 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2de63f29c4a143e19831d8c341185365 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2de63f29c4a143e19831d8c341185365" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 36235 } } peers { permanent_uuid: "e3c923fd30d944779227d74276d95d4f" member_type: VOTER last_known_addr { host: "127.27.180.189" port: 36587 } } peers { permanent_uuid: "b3fdaa1655204114938dc2ee6b906d99" member_type: VOTER last_known_addr { host: "127.27.180.188" port: 41433 } }
I20260811 18:59:43.813568 5555 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2de63f29c4a143e19831d8c341185365 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2de63f29c4a143e19831d8c341185365, State: Initialized, Role: FOLLOWER
I20260811 18:59:43.813901 5555 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2de63f29c4a143e19831d8c341185365 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2de63f29c4a143e19831d8c341185365" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 36235 } } peers { permanent_uuid: "e3c923fd30d944779227d74276d95d4f" member_type: VOTER last_known_addr { host: "127.27.180.189" port: 36587 } } peers { permanent_uuid: "b3fdaa1655204114938dc2ee6b906d99" member_type: VOTER last_known_addr { host: "127.27.180.188" port: 41433 } }
I20260811 18:59:43.814916 5742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:43.799729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43942 (local address 127.27.180.189:36587)
0811 18:59:43.799829 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:43.799833 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:43.799923 (+ 90us) server_negotiation.cc:407] Connection header received
0811 18:59:43.800080 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:43.800083 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:43.800124 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:43.800169 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:43.800931 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:43.801552 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:43.802250 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:43.802456 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:43.802537 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:43.810681 (+ 8144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:43.810684 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:43.810687 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:59:43.810721 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:43.812111 (+ 1390us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:43.813545 (+ 1434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:43.813548 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:43.813550 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:43.813632 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:43.813921 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:43.813925 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:43.813926 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:43.814111 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:43.814274 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:43.814678 (+ 404us) server_negotiation.cc:299] Negotiation successful
0811 18:59:43.814784 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260811 18:59:43.815294 5555 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2de63f29c4a143e19831d8c341185365 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:59:43.816149 5883 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b3fdaa1655204114938dc2ee6b906d99: Bootstrap starting.
I20260811 18:59:43.818145 5731 raft_consensus.cc:348] T 00000000000000000000000000000000 P e3c923fd30d944779227d74276d95d4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2de63f29c4a143e19831d8c341185365" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 36235 } } peers { permanent_uuid: "e3c923fd30d944779227d74276d95d4f" member_type: VOTER last_known_addr { host: "127.27.180.189" port: 36587 } } peers { permanent_uuid: "b3fdaa1655204114938dc2ee6b906d99" member_type: VOTER last_known_addr { host: "127.27.180.188" port: 41433 } }
I20260811 18:59:43.818372 5731 raft_consensus.cc:729] T 00000000000000000000000000000000 P e3c923fd30d944779227d74276d95d4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3c923fd30d944779227d74276d95d4f, State: Initialized, Role: FOLLOWER
I20260811 18:59:43.818619 28370 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:59:43.819063 5883 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b3fdaa1655204114938dc2ee6b906d99: Neither blocks nor log segments found. Creating new log.
I20260811 18:59:43.819895 5883 log.cc:824] T 00000000000000000000000000000000 P b3fdaa1655204114938dc2ee6b906d99: Log is configured to *not* fsync() on all Append() calls
I20260811 18:59:43.819964 5513 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "2de63f29c4a143e19831d8c341185365"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54636
I20260811 18:59:43.820266 5513 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2de63f29c4a143e19831d8c341185365 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260811 18:59:43.820381 5513 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2de63f29c4a143e19831d8c341185365 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:59:43.821609 5513 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2de63f29c4a143e19831d8c341185365 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2de63f29c4a143e19831d8c341185365" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 36235 } } peers { permanent_uuid: "e3c923fd30d944779227d74276d95d4f" member_type: VOTER last_known_addr { host: "127.27.180.189" port: 36587 } } peers { permanent_uuid: "b3fdaa1655204114938dc2ee6b906d99" member_type: VOTER last_known_addr { host: "127.27.180.188" port: 41433 } }
W20260811 18:59:43.821861 5922 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2de63f29c4a143e19831d8c341185365: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:59:43.821938 5922 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:59:43.822129 5883 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b3fdaa1655204114938dc2ee6b906d99: No bootstrap required, opened a new log
I20260811 18:59:43.822522 5513 leader_election.cc:288] T 00000000000000000000000000000000 P 2de63f29c4a143e19831d8c341185365 [CANDIDATE]: Term 1 election: Requested vote from peers e3c923fd30d944779227d74276d95d4f (127.27.180.189:36587), b3fdaa1655204114938dc2ee6b906d99 (127.27.180.188:41433)
I20260811 18:59:43.819909 5433 master_runner.cc:186] Error getting master registration for 127.27.180.189:36587: OK, instance_id { permanent_uuid: "e3c923fd30d944779227d74276d95d4f" instance_seqno: 1786474783546132 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 18:59:43.823014 5433 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260811 18:59:43.824105 5473 tablet.cc:2369] T 00000000000000000000000000000000 P 2de63f29c4a143e19831d8c341185365: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 18:59:43.824695 5731 consensus_queue.cc:260] T 00000000000000000000000000000000 P e3c923fd30d944779227d74276d95d4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2de63f29c4a143e19831d8c341185365" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 36235 } } peers { permanent_uuid: "e3c923fd30d944779227d74276d95d4f" member_type: VOTER last_known_addr { host: "127.27.180.189" port: 36587 } } peers { permanent_uuid: "b3fdaa1655204114938dc2ee6b906d99" member_type: VOTER last_known_addr { host: "127.27.180.188" port: 41433 } }
I20260811 18:59:43.825248 5938 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3c923fd30d944779227d74276d95d4f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2de63f29c4a143e19831d8c341185365" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 36235 } } peers { permanent_uuid: "e3c923fd30d944779227d74276d95d4f" member_type: VOTER last_known_addr { host: "127.27.180.189" port: 36587 } } peers { permanent_uuid: "b3fdaa1655204114938dc2ee6b906d99" member_type: VOTER last_known_addr { host: "127.27.180.188" port: 41433 } } }
I20260811 18:59:43.825448 5938 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3c923fd30d944779227d74276d95d4f [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:59:43.826444 5731 sys_catalog.cc:564] T 00000000000000000000000000000000 P e3c923fd30d944779227d74276d95d4f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:59:43.826484 5670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2de63f29c4a143e19831d8c341185365" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "e3c923fd30d944779227d74276d95d4f"
I20260811 18:59:43.826683 5670 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e3c923fd30d944779227d74276d95d4f [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:59:43.828029 5670 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e3c923fd30d944779227d74276d95d4f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2de63f29c4a143e19831d8c341185365 in term 1.
I20260811 18:59:43.827994 5883 raft_consensus.cc:348] T 00000000000000000000000000000000 P b3fdaa1655204114938dc2ee6b906d99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2de63f29c4a143e19831d8c341185365" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 36235 } } peers { permanent_uuid: "e3c923fd30d944779227d74276d95d4f" member_type: VOTER last_known_addr { host: "127.27.180.189" port: 36587 } } peers { permanent_uuid: "b3fdaa1655204114938dc2ee6b906d99" member_type: VOTER last_known_addr { host: "127.27.180.188" port: 41433 } }
I20260811 18:59:43.828437 5462 leader_election.cc:302] T 00000000000000000000000000000000 P 2de63f29c4a143e19831d8c341185365 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2de63f29c4a143e19831d8c341185365, e3c923fd30d944779227d74276d95d4f; no voters:
I20260811 18:59:43.828814 5883 raft_consensus.cc:729] T 00000000000000000000000000000000 P b3fdaa1655204114938dc2ee6b906d99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3fdaa1655204114938dc2ee6b906d99, State: Initialized, Role: FOLLOWER
I20260811 18:59:43.829077 5968 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2de63f29c4a143e19831d8c341185365 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2de63f29c4a143e19831d8c341185365" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 36235 } } peers { permanent_uuid: "e3c923fd30d944779227d74276d95d4f" member_type: VOTER last_known_addr { host: "127.27.180.189" port: 36587 } } peers { permanent_uuid: "b3fdaa1655204114938dc2ee6b906d99" member_type: VOTER last_known_addr { host: "127.27.180.188" port: 41433 } } }
I20260811 18:59:43.829233 5968 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2de63f29c4a143e19831d8c341185365 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:59:43.829319 5968 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2de63f29c4a143e19831d8c341185365 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:59:43.829384 5968 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2de63f29c4a143e19831d8c341185365 [term 1 LEADER]: Becoming Leader. State: Replica: 2de63f29c4a143e19831d8c341185365, State: Running, Role: LEADER
I20260811 18:59:43.829614 5968 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2de63f29c4a143e19831d8c341185365 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2de63f29c4a143e19831d8c341185365" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 36235 } } peers { permanent_uuid: "e3c923fd30d944779227d74276d95d4f" member_type: VOTER last_known_addr { host: "127.27.180.189" port: 36587 } } peers { permanent_uuid: "b3fdaa1655204114938dc2ee6b906d99" member_type: VOTER last_known_addr { host: "127.27.180.188" port: 41433 } }
I20260811 18:59:43.829617 5883 consensus_queue.cc:260] T 00000000000000000000000000000000 P b3fdaa1655204114938dc2ee6b906d99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2de63f29c4a143e19831d8c341185365" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 36235 } } peers { permanent_uuid: "e3c923fd30d944779227d74276d95d4f" member_type: VOTER last_known_addr { host: "127.27.180.189" port: 36587 } } peers { permanent_uuid: "b3fdaa1655204114938dc2ee6b906d99" member_type: VOTER last_known_addr { host: "127.27.180.188" port: 41433 } }
I20260811 18:59:43.831390 5968 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2de63f29c4a143e19831d8c341185365 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2de63f29c4a143e19831d8c341185365. Latest consensus state: current_term: 1 leader_uuid: "2de63f29c4a143e19831d8c341185365" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2de63f29c4a143e19831d8c341185365" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 36235 } } peers { permanent_uuid: "e3c923fd30d944779227d74276d95d4f" member_type: VOTER last_known_addr { host: "127.27.180.189" port: 36587 } } peers { permanent_uuid: "b3fdaa1655204114938dc2ee6b906d99" member_type: VOTER last_known_addr { host: "127.27.180.188" port: 41433 } } }
I20260811 18:59:43.831497 5968 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2de63f29c4a143e19831d8c341185365 [sys.catalog]: This master's current role is: LEADER
I20260811 18:59:43.831854 5883 sys_catalog.cc:564] T 00000000000000000000000000000000 P b3fdaa1655204114938dc2ee6b906d99 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:59:43.832580 5830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2de63f29c4a143e19831d8c341185365" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "b3fdaa1655204114938dc2ee6b906d99"
I20260811 18:59:43.832818 5830 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b3fdaa1655204114938dc2ee6b906d99 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:59:43.834601 5830 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b3fdaa1655204114938dc2ee6b906d99 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2de63f29c4a143e19831d8c341185365 in term 1.
W20260811 18:59:43.835160 5964 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e3c923fd30d944779227d74276d95d4f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:59:43.835244 5964 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:59:43.836213 5571 master_runner.cc:186] Error getting master registration for 127.27.180.188:41433: OK, instance_id { permanent_uuid: "b3fdaa1655204114938dc2ee6b906d99" instance_seqno: 1786474783761696 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 18:59:43.839030 5571 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 18:59:43.842466 5971 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3fdaa1655204114938dc2ee6b906d99 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2de63f29c4a143e19831d8c341185365" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 36235 } } peers { permanent_uuid: "e3c923fd30d944779227d74276d95d4f" member_type: VOTER last_known_addr { host: "127.27.180.189" port: 36587 } } peers { permanent_uuid: "b3fdaa1655204114938dc2ee6b906d99" member_type: VOTER last_known_addr { host: "127.27.180.188" port: 41433 } } }
I20260811 18:59:43.842593 5971 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3fdaa1655204114938dc2ee6b906d99 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:59:43.843571 5891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:43.830198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34700 (local address 127.27.180.188:41433)
0811 18:59:43.830336 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:43.830340 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:43.834272 (+ 3932us) server_negotiation.cc:407] Connection header received
0811 18:59:43.834454 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:43.834458 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:43.834503 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:43.834549 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:43.835581 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:43.836309 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:43.837493 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:43.837817 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:43.837910 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:43.839920 (+ 2010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:43.839924 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:43.839926 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:59:43.839953 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:43.841585 (+ 1632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:43.842137 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:43.842141 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:43.842142 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:43.842243 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:43.842559 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:43.842562 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:43.842564 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:43.842776 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:43.842874 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:43.843311 (+ 437us) server_negotiation.cc:299] Negotiation successful
0811 18:59:43.843427 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
W20260811 18:59:43.845496 5998 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b3fdaa1655204114938dc2ee6b906d99: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:59:43.845573 5998 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:59:43.847219 5977 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:59:43.848124 5977 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:59:43.849850 5742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:43.829740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43948 (local address 127.27.180.189:36587)
0811 18:59:43.829842 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:43.829847 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:43.830586 (+ 739us) server_negotiation.cc:407] Connection header received
0811 18:59:43.830782 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:43.830785 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:43.830831 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:43.830876 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:43.831820 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:43.832439 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:43.840544 (+ 8105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:43.840892 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:43.840976 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:43.845390 (+ 4414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:43.845394 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:43.845396 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:59:43.845421 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:43.847014 (+ 1593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:43.848375 (+ 1361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:43.848379 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:43.848380 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:43.848471 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:43.848800 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:43.848803 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:43.848805 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:43.849028 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:43.849181 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:43.849549 (+ 368us) server_negotiation.cc:299] Negotiation successful
0811 18:59:43.849684 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 18:59:43.853761 5557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:43.830713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54648 (local address 127.27.180.190:36235)
0811 18:59:43.830818 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:43.830822 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:43.838585 (+ 7763us) server_negotiation.cc:407] Connection header received
0811 18:59:43.838751 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:43.838755 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:43.838804 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:43.838853 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:43.839803 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:43.840468 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:43.841490 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:43.841781 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:43.841852 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:43.848356 (+ 6504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:43.848360 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:43.848363 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:59:43.848398 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:43.849963 (+ 1565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:43.852359 (+ 2396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:43.852363 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:43.852387 (+ 24us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:43.852477 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:43.852793 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:43.852796 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:43.852798 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:43.853008 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:43.853136 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:43.853496 (+ 360us) server_negotiation.cc:299] Negotiation successful
0811 18:59:43.853614 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 18:59:43.855080 5670 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e3c923fd30d944779227d74276d95d4f [term 1 FOLLOWER]: Refusing update from remote peer 2de63f29c4a143e19831d8c341185365: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:59:43.855099 5830 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b3fdaa1655204114938dc2ee6b906d99 [term 1 FOLLOWER]: Refusing update from remote peer 2de63f29c4a143e19831d8c341185365: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:59:43.855726 5968 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2de63f29c4a143e19831d8c341185365 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3fdaa1655204114938dc2ee6b906d99" member_type: VOTER last_known_addr { host: "127.27.180.188" port: 41433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:59:43.856127 5902 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2de63f29c4a143e19831d8c341185365 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3c923fd30d944779227d74276d95d4f" member_type: VOTER last_known_addr { host: "127.27.180.189" port: 36587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:59:43.859827 5938 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3c923fd30d944779227d74276d95d4f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2de63f29c4a143e19831d8c341185365. Latest consensus state: current_term: 1 leader_uuid: "2de63f29c4a143e19831d8c341185365" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2de63f29c4a143e19831d8c341185365" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 36235 } } peers { permanent_uuid: "e3c923fd30d944779227d74276d95d4f" member_type: VOTER last_known_addr { host: "127.27.180.189" port: 36587 } } peers { permanent_uuid: "b3fdaa1655204114938dc2ee6b906d99" member_type: VOTER last_known_addr { host: "127.27.180.188" port: 41433 } } }
I20260811 18:59:43.859939 5938 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3c923fd30d944779227d74276d95d4f [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:59:43.860080 5971 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3fdaa1655204114938dc2ee6b906d99 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2de63f29c4a143e19831d8c341185365. Latest consensus state: current_term: 1 leader_uuid: "2de63f29c4a143e19831d8c341185365" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2de63f29c4a143e19831d8c341185365" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 36235 } } peers { permanent_uuid: "e3c923fd30d944779227d74276d95d4f" member_type: VOTER last_known_addr { host: "127.27.180.189" port: 36587 } } peers { permanent_uuid: "b3fdaa1655204114938dc2ee6b906d99" member_type: VOTER last_known_addr { host: "127.27.180.188" port: 41433 } } }
I20260811 18:59:43.860198 5971 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3fdaa1655204114938dc2ee6b906d99 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:59:43.865628 5902 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2de63f29c4a143e19831d8c341185365 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2de63f29c4a143e19831d8c341185365" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2de63f29c4a143e19831d8c341185365" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 36235 } } peers { permanent_uuid: "e3c923fd30d944779227d74276d95d4f" member_type: VOTER last_known_addr { host: "127.27.180.189" port: 36587 } } peers { permanent_uuid: "b3fdaa1655204114938dc2ee6b906d99" member_type: VOTER last_known_addr { host: "127.27.180.188" port: 41433 } } }
I20260811 18:59:43.865753 5902 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2de63f29c4a143e19831d8c341185365 [sys.catalog]: This master's current role is: LEADER
I20260811 18:59:43.866784 5971 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3fdaa1655204114938dc2ee6b906d99 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2de63f29c4a143e19831d8c341185365" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2de63f29c4a143e19831d8c341185365" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 36235 } } peers { permanent_uuid: "e3c923fd30d944779227d74276d95d4f" member_type: VOTER last_known_addr { host: "127.27.180.189" port: 36587 } } peers { permanent_uuid: "b3fdaa1655204114938dc2ee6b906d99" member_type: VOTER last_known_addr { host: "127.27.180.188" port: 41433 } } }
I20260811 18:59:43.866900 5971 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3fdaa1655204114938dc2ee6b906d99 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:59:43.867208 6004 mvcc.cc:205] Tried to move back new op lower bound from 7317400714664058880 to 7317400714569306112. Current Snapshot: MvccSnapshot[applied={T|T < 7317400714664058880}]
I20260811 18:59:43.867396 6000 mvcc.cc:205] Tried to move back new op lower bound from 7317400714664058880 to 7317400714569306112. Current Snapshot: MvccSnapshot[applied={T|T < 7317400714664058880 or (T in {7317400714664058880})}]
I20260811 18:59:43.867633 5968 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2de63f29c4a143e19831d8c341185365 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2de63f29c4a143e19831d8c341185365" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2de63f29c4a143e19831d8c341185365" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 36235 } } peers { permanent_uuid: "e3c923fd30d944779227d74276d95d4f" member_type: VOTER last_known_addr { host: "127.27.180.189" port: 36587 } } peers { permanent_uuid: "b3fdaa1655204114938dc2ee6b906d99" member_type: VOTER last_known_addr { host: "127.27.180.188" port: 41433 } } }
I20260811 18:59:43.867743 5968 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2de63f29c4a143e19831d8c341185365 [sys.catalog]: This master's current role is: LEADER
I20260811 18:59:43.868489 5938 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3c923fd30d944779227d74276d95d4f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2de63f29c4a143e19831d8c341185365" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2de63f29c4a143e19831d8c341185365" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 36235 } } peers { permanent_uuid: "e3c923fd30d944779227d74276d95d4f" member_type: VOTER last_known_addr { host: "127.27.180.189" port: 36587 } } peers { permanent_uuid: "b3fdaa1655204114938dc2ee6b906d99" member_type: VOTER last_known_addr { host: "127.27.180.188" port: 41433 } } }
I20260811 18:59:43.868597 5938 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3c923fd30d944779227d74276d95d4f [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:59:43.869072 5977 catalog_manager.cc:1347] Generated new cluster ID: 08c2578ac5fe4255ac9407329dfc944e
I20260811 18:59:43.869155 5977 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:59:43.924268 5977 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:59:43.925631 5977 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:59:43.940837 5977 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2de63f29c4a143e19831d8c341185365: Generated new TSK 0
I20260811 18:59:43.941708 5977 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 18:59:43.973498 5557 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:59:43.970294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54658 (local address 127.27.180.190:36235)
0811 18:59:43.970509 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:43.970514 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:43.972221 (+ 1707us) server_negotiation.cc:407] Connection header received
0811 18:59:43.972463 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:43.972468 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:43.972526 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:43.972588 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:43.973355 (+ 767us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54658: BlockingRecv error: recv got EOF from 127.0.0.1:54658 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260811 18:59:43.975118 5891 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:59:43.971505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34706 (local address 127.27.180.188:41433)
0811 18:59:43.971645 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:43.971650 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:43.974077 (+ 2427us) server_negotiation.cc:407] Connection header received
0811 18:59:43.974273 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:43.974277 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:43.974331 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:43.974388 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:43.974987 (+ 599us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34706: BlockingRecv error: recv got EOF from 127.0.0.1:34706 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260811 18:59:43.977399 5742 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:59:43.975701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43956 (local address 127.27.180.189:36587)
0811 18:59:43.975833 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:43.975838 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:43.976254 (+ 416us) server_negotiation.cc:407] Connection header received
0811 18:59:43.976553 (+ 299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:43.976557 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:43.976611 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:43.976680 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:43.977264 (+ 584us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43956: BlockingRecv error: recv got EOF from 127.0.0.1:43956 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260811 18:59:43.982337 5742 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:59:43.980654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43960 (local address 127.27.180.189:36587)
0811 18:59:43.980777 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:43.980782 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:43.981315 (+ 533us) server_negotiation.cc:407] Connection header received
0811 18:59:43.981543 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:43.981547 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:43.981627 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:43.981696 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:43.982227 (+ 531us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43960: BlockingRecv error: recv got EOF from 127.0.0.1:43960 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260811 18:59:43.987846 5742 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:59:43.986192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43970 (local address 127.27.180.189:36587)
0811 18:59:43.986305 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:43.986310 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:43.986844 (+ 534us) server_negotiation.cc:407] Connection header received
0811 18:59:43.987052 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:43.987056 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:43.987114 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:43.987172 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:43.987738 (+ 566us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43970: BlockingRecv error: recv got EOF from 127.0.0.1:43970 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260811 18:59:43.994304 5742 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:59:43.992521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43974 (local address 127.27.180.189:36587)
0811 18:59:43.992644 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:43.992648 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:43.993248 (+ 600us) server_negotiation.cc:407] Connection header received
0811 18:59:43.993505 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:43.993508 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:43.993565 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:43.993624 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:43.994199 (+ 575us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43974: BlockingRecv error: recv got EOF from 127.0.0.1:43974 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260811 18:59:44.002748 5742 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:59:44.000948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43980 (local address 127.27.180.189:36587)
0811 18:59:44.001115 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:44.001120 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:44.001655 (+ 535us) server_negotiation.cc:407] Connection header received
0811 18:59:44.001902 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:44.001906 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:44.001965 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:44.002025 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:44.002621 (+ 596us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43980: BlockingRecv error: recv got EOF from 127.0.0.1:43980 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
W20260811 18:59:44.015381 5742 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:59:44.013520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43984 (local address 127.27.180.189:36587)
0811 18:59:44.013644 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:44.013649 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:44.014236 (+ 587us) server_negotiation.cc:407] Connection header received
0811 18:59:44.014449 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:44.014453 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:44.014510 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:44.014566 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:44.015263 (+ 697us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43984: BlockingRecv error: recv got EOF from 127.0.0.1:43984 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260811 18:59:44.036047 5742 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:59:44.034456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43994 (local address 127.27.180.189:36587)
0811 18:59:44.034595 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:44.034601 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:44.035153 (+ 552us) server_negotiation.cc:407] Connection header received
0811 18:59:44.035314 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:44.035318 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:44.035376 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:44.035452 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:44.035935 (+ 483us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43994: BlockingRecv error: recv got EOF from 127.0.0.1:43994 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260811 18:59:44.071616 5742 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:59:44.070496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43996 (local address 127.27.180.189:36587)
0811 18:59:44.070647 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:44.070652 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:44.070789 (+ 137us) server_negotiation.cc:407] Connection header received
0811 18:59:44.070930 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:44.070934 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:44.070990 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:44.071088 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:44.071499 (+ 411us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43996: BlockingRecv error: recv got EOF from 127.0.0.1:43996 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260811 18:59:44.138875 5742 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:59:44.137773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44006 (local address 127.27.180.189:36587)
0811 18:59:44.137947 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:44.137952 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:44.138045 (+ 93us) server_negotiation.cc:407] Connection header received
0811 18:59:44.138183 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:44.138186 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:44.138242 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:44.138316 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:44.138748 (+ 432us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44006: BlockingRecv error: recv got EOF from 127.0.0.1:44006 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
W20260811 18:59:44.270321 5742 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:59:44.269209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44018 (local address 127.27.180.189:36587)
0811 18:59:44.269344 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:44.269348 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:44.269575 (+ 227us) server_negotiation.cc:407] Connection header received
0811 18:59:44.269696 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:44.269698 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:44.269745 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:44.269825 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:44.270202 (+ 377us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44018: BlockingRecv error: recv got EOF from 127.0.0.1:44018 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260811 18:59:44.529492 5742 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:59:44.528444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44026 (local address 127.27.180.189:36587)
0811 18:59:44.528569 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:44.528575 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:44.528724 (+ 149us) server_negotiation.cc:407] Connection header received
0811 18:59:44.528826 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:44.528829 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:44.528877 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:44.528949 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:44.529370 (+ 421us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44026: BlockingRecv error: recv got EOF from 127.0.0.1:44026 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 18:59:44.836503 5964 catalog_manager.cc:1259] Loaded cluster ID: 08c2578ac5fe4255ac9407329dfc944e
I20260811 18:59:44.836588 5964 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e3c923fd30d944779227d74276d95d4f: loading cluster ID for follower catalog manager: success
I20260811 18:59:44.838284 5964 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e3c923fd30d944779227d74276d95d4f: acquiring CA information for follower catalog manager: success
I20260811 18:59:44.839470 5964 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e3c923fd30d944779227d74276d95d4f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 18:59:44.846892 5998 catalog_manager.cc:1259] Loaded cluster ID: 08c2578ac5fe4255ac9407329dfc944e
I20260811 18:59:44.846951 5998 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b3fdaa1655204114938dc2ee6b906d99: loading cluster ID for follower catalog manager: success
I20260811 18:59:44.848448 5998 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b3fdaa1655204114938dc2ee6b906d99: acquiring CA information for follower catalog manager: success
I20260811 18:59:44.849529 5998 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b3fdaa1655204114938dc2ee6b906d99: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 18:59:45.047487 6143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:45.043646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44030 (local address 127.27.180.189:36587)
0811 18:59:45.043879 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:45.043883 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:45.043928 (+ 45us) server_negotiation.cc:407] Connection header received
0811 18:59:45.044045 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:45.044047 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:45.044173 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:45.044270 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:59:45.044612 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:45.045162 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:45.046035 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:45.046263 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:45.046353 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:45.046656 (+ 303us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:59:45.047080 (+ 424us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:59:45.047189 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:45.047268 (+ 79us) server_negotiation.cc:299] Negotiation successful
0811 18:59:45.047325 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":73,"threads_started":1}
I20260811 18:59:45.053639 6155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:45.050481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34712 (local address 127.27.180.188:41433)
0811 18:59:45.050714 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:45.050717 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:45.050730 (+ 13us) server_negotiation.cc:407] Connection header received
0811 18:59:45.050808 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:45.050810 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:45.050926 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:45.051016 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:59:45.051160 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:45.051648 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:45.052387 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:45.052599 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:45.052666 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:45.052877 (+ 211us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:59:45.053297 (+ 420us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:59:45.053380 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:45.053437 (+ 57us) server_negotiation.cc:299] Negotiation successful
0811 18:59:45.053485 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":82,"threads_started":1}
I20260811 18:59:45.060391 6143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:45.056609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44036 (local address 127.27.180.189:36587)
0811 18:59:45.056745 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:45.056749 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:45.056761 (+ 12us) server_negotiation.cc:407] Connection header received
0811 18:59:45.056844 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:45.056847 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:45.056897 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:45.056955 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:59:45.057174 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:45.057907 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:45.059061 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:45.059427 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:45.059525 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:45.059782 (+ 257us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:59:45.059862 (+ 80us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:59:45.059928 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:45.060194 (+ 266us) server_negotiation.cc:299] Negotiation successful
0811 18:59:45.060250 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 18:59:45.060890 6155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:45.056418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34720 (local address 127.27.180.188:41433)
0811 18:59:45.056530 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:45.056534 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:45.057229 (+ 695us) server_negotiation.cc:407] Connection header received
0811 18:59:45.057329 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:45.057332 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:45.057389 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:45.057455 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:59:45.057775 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:45.058507 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:45.059622 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:45.059923 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:45.060010 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:45.060272 (+ 262us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:59:45.060338 (+ 66us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:59:45.060391 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:45.060698 (+ 307us) server_negotiation.cc:299] Negotiation successful
0811 18:59:45.060756 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260811 18:59:45.061339 6166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:45.056346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54664 (local address 127.27.180.190:36235)
0811 18:59:45.056803 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:45.056807 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:45.056916 (+ 109us) server_negotiation.cc:407] Connection header received
0811 18:59:45.057061 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:45.057065 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:45.057260 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:45.057386 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:59:45.057559 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:45.058230 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:45.059227 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:45.059479 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:45.059572 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:45.059951 (+ 379us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:59:45.060296 (+ 345us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:59:45.060395 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:45.060473 (+ 78us) server_negotiation.cc:299] Negotiation successful
0811 18:59:45.061200 (+ 727us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":376,"thread_start_us":88,"threads_started":1}
I20260811 18:59:45.063973 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 5433
I20260811 18:59:45.083878 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 5571
I20260811 18:59:45.098201 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 5746
2026-08-11T18:59:45Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3249 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474496157106-28370-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 18:59:45 dist-test-slave-2j7r krb5kdc[6176](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 18:59:45 dist-test-slave-2j7r krb5kdc[6176](info): set up 2 sockets
Aug 11 18:59:45 dist-test-slave-2j7r krb5kdc[6176](info): commencing operation
krb5kdc: starting...
W20260811 18:59:46.285915 28370 mini_kdc.cc:121] Time spent starting KDC: real 1.163s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474496157106-28370-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474496157106-28370-0/krb5kdc/test-user.keytab.
Aug 11 18:59:46 dist-test-slave-2j7r krb5kdc[6176](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474786, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T18:59:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T18:59:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.27.180.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.180.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.180.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab.
Entry for principal kudu/127.27.180.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab.
I20260811 18:59:46.403679 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474496157106-28370-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474496157106-28370-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474496157106-28370-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.190:40049
--webserver_interface=127.27.180.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab
--principal=kudu/127.27.180.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:44375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.180.190:40049,127.27.180.189:44959,127.27.180.188:43013
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 18:59:46.514935 6200 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 18:59:46.515184 6200 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:59:46.515205 6200 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:59:46.515239 6200 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:59:46.519063 6200 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:59:46.519106 6200 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:59:46.519124 6200 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:59:46.519143 6200 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:59:46.519160 6200 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:59:46.524149 6200 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:44375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474496157106-28370-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474496157106-28370-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.27.180.190:40049,127.27.180.189:44959,127.27.180.188:43013
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.190.keytab
--principal=kudu/127.27.180.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.190:40049
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.27.180.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6200
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474496157106-28370-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:59:46.525027 6200 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:59:46.525957 6200 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:59:46.533977 6214 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:59:46.534044 6200 server_base.cc:1034] running on GCE node
W20260811 18:59:46.534098 6219 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:46.533994 6216 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:59:46.534714 6200 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:59:46.535699 6200 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:59:46.536840 6200 hybrid_clock.cc:648] HybridClock initialized: now 1786474786536823 us; error 34 us; skew 500 ppm
Aug 11 18:59:46 dist-test-slave-2j7r krb5kdc[6176](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474786, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:59:46.542325 6200 init.cc:377] Logged in from keytab as kudu/127.27.180.190@KRBTEST.COM (short username kudu)
I20260811 18:59:46.543141 6200 webserver.cc:460] Webserver started at http://127.27.180.190:41061/ using document root <none> and password file <none>
I20260811 18:59:46.543632 6200 fs_manager.cc:359] Metadata directory not provided
I20260811 18:59:46.543686 6200 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:59:46.543848 6200 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:59:46.545796 6200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474496157106-28370-0/minicluster-data/master-0/data/instance:
uuid: "c150e5b86c96445c8d6e1bdf7dcb0d21"
format_stamp: "Formatted at 2026-08-11 18:59:46 on dist-test-slave-2j7r"
server_key: "63596d5c330fb8ed550392b36d50ccdf"
server_key_iv: "a5b3c6394be3da8514506ad60e7264d0"
server_key_version: "encryptionkey@0"
I20260811 18:59:46.546329 6200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474496157106-28370-0/minicluster-data/master-0/wal/instance:
uuid: "c150e5b86c96445c8d6e1bdf7dcb0d21"
format_stamp: "Formatted at 2026-08-11 18:59:46 on dist-test-slave-2j7r"
server_key: "63596d5c330fb8ed550392b36d50ccdf"
server_key_iv: "a5b3c6394be3da8514506ad60e7264d0"
server_key_version: "encryptionkey@0"
I20260811 18:59:46.549863 6200 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260811 18:59:46.552132 6233 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:46.553318 6200 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 18:59:46.553438 6200 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474496157106-28370-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474496157106-28370-0/minicluster-data/master-0/wal
uuid: "c150e5b86c96445c8d6e1bdf7dcb0d21"
format_stamp: "Formatted at 2026-08-11 18:59:46 on dist-test-slave-2j7r"
server_key: "63596d5c330fb8ed550392b36d50ccdf"
server_key_iv: "a5b3c6394be3da8514506ad60e7264d0"
server_key_version: "encryptionkey@0"
I20260811 18:59:46.553527 6200 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474496157106-28370-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474496157106-28370-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474496157106-28370-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:59:46.576707 6200 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:59:46.579566 6200 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:59:46.579726 6200 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:59:46.586552 6200 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.190:40049
I20260811 18:59:46.586812 6343 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.190:40049 every 8 connection(s)
I20260811 18:59:46.587468 6200 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474496157106-28370-0/minicluster-data/master-0/data/info.pb
I20260811 18:59:46.590180 6350 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:59:46.590350 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 6200
I20260811 18:59:46.590466 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474496157106-28370-0/minicluster-data/master-0/wal/instance
I20260811 18:59:46.590725 28370 external_mini_cluster.cc:1428] Setting key 49734776192592c77f29b899477ae6f5
I20260811 18:59:46.593881 6350 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.180.190" port: 40049 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.27.180.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.180.189@KRBTEST.COM" created.
Aug 11 18:59:46 dist-test-slave-2j7r krb5kdc[6176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474786, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.190@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM
I20260811 18:59:46.608064 6364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:46.595973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55746 (local address 127.27.180.190:40049)
0811 18:59:46.596544 (+ 571us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:46.596562 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:46.596699 (+ 137us) server_negotiation.cc:407] Connection header received
0811 18:59:46.597746 (+ 1047us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:46.597752 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:46.598009 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:46.598207 (+ 198us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:46.598907 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:46.599933 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:46.600962 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:46.601363 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:46.601476 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:46.603749 (+ 2273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:46.603754 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:46.603763 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:59:46.603783 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:46.605190 (+ 1407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:46.605625 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:46.605631 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:46.605632 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:46.605728 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:46.606030 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:46.606033 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:46.606035 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:46.606380 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:46.606497 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:46.606845 (+ 348us) server_negotiation.cc:299] Negotiation successful
0811 18:59:46.607057 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":447,"thread_start_us":179,"threads_started":1}
I20260811 18:59:46.608064 6363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:46.595972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:40049 (local address 127.0.0.1:55746)
0811 18:59:46.596543 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:46.596590 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 18:59:46.597442 (+ 852us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:59:46.598192 (+ 750us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:59:46.598196 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:59:46.598523 (+ 327us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:59:46.598759 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:46.598772 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:46.600038 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:46.600040 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:46.600866 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:46.600876 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:46.601509 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:46.601512 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:46.601595 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:46.602086 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:59:46.602107 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:59:46.603616 (+ 1509us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:59:46.605290 (+ 1674us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:59:46.605298 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:59:46.605308 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:59:46.605518 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:59:46.605813 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:59:46.605816 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:59:46.605818 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:59:46.605946 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:59:46.606488 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:59:46.606494 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:59:46.606623 (+ 129us) client_negotiation.cc:769] Sending connection context
0811 18:59:46.606765 (+ 142us) client_negotiation.cc:240] Negotiation successful
0811 18:59:46.607146 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":336,"mutex_wait_us":92,"thread_start_us":113,"threads_started":1}
I20260811 18:59:46.610335 6350 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.180.189" port: 44959 } has no permanent_uuid. Determining permanent_uuid...
I20260811 18:59:46.611001 6363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:46.610783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.189:44959 (local address 127.0.0.1:33600)
0811 18:59:46.610873 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:46.610939 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.189:44959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260811 18:59:46.611213 6236 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.27.180.189:44959: connect: Connection refused (error 111)
I20260811 18:59:46.612011 6363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:46.611803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.189:44959 (local address 127.0.0.1:33606)
0811 18:59:46.611900 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:46.611947 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.189:44959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260811 18:59:46.612135 6350 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.180.189:44959: Network error: Client connection negotiation failed: client connection to 127.27.180.189:44959: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.180.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.189.keytab.
Entry for principal kudu/127.27.180.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.189.keytab.
I20260811 18:59:46.615721 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474496157106-28370-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474496157106-28370-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474496157106-28370-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474496157106-28370-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.189:44959
--webserver_interface=127.27.180.189
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.189.keytab
--principal=kudu/127.27.180.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:44375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.180.190:40049,127.27.180.189:44959,127.27.180.188:43013
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 18:59:46.661374 6350 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.180.189" port: 44959 } attempt: 1
I20260811 18:59:46.662240 6363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:46.661942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.189:44959 (local address 127.0.0.1:33618)
0811 18:59:46.662096 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:46.662159 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.189:44959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260811 18:59:46.662945 6363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:46.662718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.189:44959 (local address 127.0.0.1:33626)
0811 18:59:46.662836 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:46.662883 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.189:44959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260811 18:59:46.663075 6350 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.180.189:44959: Network error: Client connection negotiation failed: client connection to 127.27.180.189:44959: connect: Connection refused (error 111)
W20260811 18:59:46.724083 6373 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 18:59:46.724337 6373 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:59:46.724378 6373 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:59:46.724438 6373 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:59:46.728323 6373 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:59:46.728376 6373 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:59:46.728403 6373 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:59:46.728425 6373 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:59:46.728453 6373 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:59:46.731225 6350 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.180.189" port: 44959 } attempt: 2
I20260811 18:59:46.731990 6363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:46.731711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.189:44959 (local address 127.0.0.1:33638)
0811 18:59:46.731853 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:46.731913 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.189:44959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260811 18:59:46.732646 6363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:46.732433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.189:44959 (local address 127.0.0.1:33646)
0811 18:59:46.732540 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:46.732586 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.189:44959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260811 18:59:46.732790 6350 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.180.189:44959: Network error: Client connection negotiation failed: client connection to 127.27.180.189:44959: connect: Connection refused (error 111)
I20260811 18:59:46.733719 6373 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:44375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474496157106-28370-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474496157106-28370-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.27.180.190:40049,127.27.180.189:44959,127.27.180.188:43013
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.189.keytab
--principal=kudu/127.27.180.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.189:44959
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474496157106-28370-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.27.180.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6373
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474496157106-28370-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:59:46.734622 6373 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:59:46.735527 6373 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 18:59:46.743564 6386 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:46.743573 6384 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:46.743573 6383 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:59:46.743721 6373 server_base.cc:1034] running on GCE node
I20260811 18:59:46.744268 6373 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:59:46.745266 6373 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:59:46.746421 6373 hybrid_clock.cc:648] HybridClock initialized: now 1786474786746399 us; error 34 us; skew 500 ppm
Aug 11 18:59:46 dist-test-slave-2j7r krb5kdc[6176](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474786, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:59:46.748890 6373 init.cc:377] Logged in from keytab as kudu/127.27.180.189@KRBTEST.COM (short username kudu)
I20260811 18:59:46.749744 6373 webserver.cc:460] Webserver started at http://127.27.180.189:46761/ using document root <none> and password file <none>
I20260811 18:59:46.750238 6373 fs_manager.cc:359] Metadata directory not provided
I20260811 18:59:46.750301 6373 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:59:46.750499 6373 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:59:46.752420 6373 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474496157106-28370-0/minicluster-data/master-1/data/instance:
uuid: "76d6f7110b1c42849a4198f8257dde74"
format_stamp: "Formatted at 2026-08-11 18:59:46 on dist-test-slave-2j7r"
server_key: "d1cd543e6ed852b9a0c74bba8db98c7b"
server_key_iv: "d06cfbd638e0264a3698daec98122e5c"
server_key_version: "encryptionkey@0"
I20260811 18:59:46.752925 6373 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474496157106-28370-0/minicluster-data/master-1/wal/instance:
uuid: "76d6f7110b1c42849a4198f8257dde74"
format_stamp: "Formatted at 2026-08-11 18:59:46 on dist-test-slave-2j7r"
server_key: "d1cd543e6ed852b9a0c74bba8db98c7b"
server_key_iv: "d06cfbd638e0264a3698daec98122e5c"
server_key_version: "encryptionkey@0"
I20260811 18:59:46.756346 6373 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.002s
I20260811 18:59:46.758651 6403 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:46.759757 6373 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 18:59:46.759879 6373 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474496157106-28370-0/minicluster-data/master-1/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474496157106-28370-0/minicluster-data/master-1/wal
uuid: "76d6f7110b1c42849a4198f8257dde74"
format_stamp: "Formatted at 2026-08-11 18:59:46 on dist-test-slave-2j7r"
server_key: "d1cd543e6ed852b9a0c74bba8db98c7b"
server_key_iv: "d06cfbd638e0264a3698daec98122e5c"
server_key_version: "encryptionkey@0"
I20260811 18:59:46.759966 6373 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474496157106-28370-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474496157106-28370-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474496157106-28370-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:59:46.772011 6373 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:59:46.774813 6373 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:59:46.774977 6373 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:59:46.781657 6373 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.189:44959
I20260811 18:59:46.781908 6510 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.189:44959 every 8 connection(s)
I20260811 18:59:46.782538 6373 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474496157106-28370-0/minicluster-data/master-1/data/info.pb
I20260811 18:59:46.785145 6511 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:59:46.788702 6511 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.180.190" port: 40049 } has no permanent_uuid. Determining permanent_uuid...
I20260811 18:59:46.791474 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 6373
I20260811 18:59:46.791574 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474496157106-28370-0/minicluster-data/master-1/wal/instance
I20260811 18:59:46.791832 28370 external_mini_cluster.cc:1428] Setting key fbe77e1444f278938aed6190a793a651
Aug 11 18:59:46 dist-test-slave-2j7r krb5kdc[6176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474786, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.189@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.27.180.188@KRBTEST.COM; defaulting to no policy
I20260811 18:59:46.801128 6364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:46.790401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35670 (local address 127.27.180.190:40049)
0811 18:59:46.790516 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:46.790520 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:46.791274 (+ 754us) server_negotiation.cc:407] Connection header received
0811 18:59:46.792202 (+ 928us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:46.792206 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:46.792273 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:46.792346 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:46.793479 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:46.793999 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:46.795086 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:46.795322 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:46.795413 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:46.797781 (+ 2368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:46.797784 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:46.797786 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:59:46.797810 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:46.799097 (+ 1287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:46.799579 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:46.799583 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:46.799585 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:46.799674 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:46.799991 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:46.799995 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:46.799997 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:46.800174 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:46.800262 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:46.800796 (+ 534us) server_negotiation.cc:299] Negotiation successful
0811 18:59:46.800933 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 18:59:46.801959 6515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:46.790694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:40049 (local address 127.0.0.1:35670)
0811 18:59:46.791132 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:46.791172 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 18:59:46.792002 (+ 830us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:59:46.792519 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:59:46.792526 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:59:46.792851 (+ 325us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:59:46.793328 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:46.793342 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:46.794118 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:46.794121 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:46.794966 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:46.794975 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:46.795466 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:46.795470 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:46.795626 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:46.796140 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:59:46.796157 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:59:46.797622 (+ 1465us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:59:46.799208 (+ 1586us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:59:46.799214 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:59:46.799228 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:59:46.799485 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:59:46.799759 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:59:46.799763 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:59:46.799765 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:59:46.799902 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:59:46.800276 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:59:46.800283 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:59:46.800558 (+ 275us) client_negotiation.cc:769] Sending connection context
0811 18:59:46.800723 (+ 165us) client_negotiation.cc:240] Negotiation successful
0811 18:59:46.801029 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":135,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.27.180.188@KRBTEST.COM" created.
I20260811 18:59:46.803393 6511 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.180.189" port: 44959 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 18:59:46 dist-test-slave-2j7r krb5kdc[6176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474786, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.189@KRBTEST.COM for kudu/127.27.180.189@KRBTEST.COM
I20260811 18:59:46.811723 6515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:46.803907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.189:44959 (local address 127.0.0.1:33662)
0811 18:59:46.804019 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:46.804031 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 18:59:46.804092 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:59:46.804653 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:59:46.804656 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:59:46.804810 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:59:46.804884 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:46.804890 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:46.805836 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:46.805839 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:46.806420 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:46.806429 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:46.806976 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:46.806980 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:46.807061 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:46.807502 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:59:46.807513 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:59:46.808578 (+ 1065us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:59:46.810299 (+ 1721us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:59:46.810302 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:59:46.810305 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:59:46.810490 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:59:46.810775 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:59:46.810778 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:59:46.810780 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:59:46.810853 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:59:46.811269 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:59:46.811272 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:59:46.811368 (+ 96us) client_negotiation.cc:769] Sending connection context
0811 18:59:46.811455 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 18:59:46.811575 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260811 18:59:46.811829 6521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:46.804048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33662 (local address 127.27.180.189:44959)
0811 18:59:46.804341 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:46.804349 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:46.804369 (+ 20us) server_negotiation.cc:407] Connection header received
0811 18:59:46.804400 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:46.804404 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:46.804542 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:46.804656 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:46.804984 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:46.805730 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:46.806524 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:46.806874 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:46.806964 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:46.808696 (+ 1732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:46.808705 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:46.808715 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:59:46.808735 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:46.810201 (+ 1466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:46.810586 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:46.810591 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:46.810592 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:46.810664 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:46.810931 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:46.810934 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:46.810936 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:46.811152 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:46.811277 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:46.811546 (+ 269us) server_negotiation.cc:299] Negotiation successful
0811 18:59:46.811667 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":215,"thread_start_us":130,"threads_started":1}
I20260811 18:59:46.813614 6511 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.180.188" port: 43013 } has no permanent_uuid. Determining permanent_uuid...
I20260811 18:59:46.814255 6515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:46.814005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.188:43013 (local address 127.0.0.1:36762)
0811 18:59:46.814099 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:46.814200 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.188:43013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260811 18:59:46.814414 6407 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.27.180.188:43013: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.27.180.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.188.keytab.
Entry for principal kudu/127.27.180.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.188.keytab.
I20260811 18:59:46.815204 6515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:46.814990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.188:43013 (local address 127.0.0.1:36774)
0811 18:59:46.815096 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:46.815143 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.188:43013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260811 18:59:46.815320 6511 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.180.188:43013: Network error: Client connection negotiation failed: client connection to 127.27.180.188:43013: connect: Connection refused (error 111)
I20260811 18:59:46.815874 28370 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
/tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474496157106-28370-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474496157106-28370-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474496157106-28370-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474496157106-28370-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.188:43013
--webserver_interface=127.27.180.188
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.188.keytab
--principal=kudu/127.27.180.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.27.180.148:44375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.27.180.190:40049,127.27.180.189:44959,127.27.180.188:43013
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474496157106-28370-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474496157106-28370-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474496157106-28370-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 18:59:46.823944 6350 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.180.189" port: 44959 } attempt: 3
Aug 11 18:59:46 dist-test-slave-2j7r krb5kdc[6176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474786, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.190@KRBTEST.COM for kudu/127.27.180.189@KRBTEST.COM
I20260811 18:59:46.832911 6521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:46.824468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33678 (local address 127.27.180.189:44959)
0811 18:59:46.824587 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:46.824590 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:46.824604 (+ 14us) server_negotiation.cc:407] Connection header received
0811 18:59:46.824744 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:46.824746 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:46.824783 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:46.824844 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:46.825348 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:46.825837 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:46.827398 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:46.828119 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:46.828200 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:46.830242 (+ 2042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:46.830245 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:46.830247 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:59:46.830270 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:46.831504 (+ 1234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:46.831940 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:46.831942 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:46.831944 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:46.832009 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:46.832244 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:46.832246 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:46.832247 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:46.832399 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:46.832467 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:46.832698 (+ 231us) server_negotiation.cc:299] Negotiation successful
0811 18:59:46.832786 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 18:59:46.832967 6363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:46.824402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.189:44959 (local address 127.0.0.1:33678)
0811 18:59:46.824543 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:46.824558 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 18:59:46.824658 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:59:46.824890 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:59:46.824893 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:59:46.825081 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:59:46.825245 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:46.825254 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:46.825955 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:46.825959 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:46.827298 (+ 1339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:46.827310 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:46.828225 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:46.828229 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:46.828338 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:46.828749 (+ 411us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:59:46.828763 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:59:46.830114 (+ 1351us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:59:46.831605 (+ 1491us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:59:46.831608 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:59:46.831610 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:59:46.831856 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:59:46.832077 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:59:46.832080 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:59:46.832082 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:59:46.832168 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:59:46.832470 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:59:46.832473 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:59:46.832571 (+ 98us) client_negotiation.cc:769] Sending connection context
0811 18:59:46.832667 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 18:59:46.832798 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260811 18:59:46.834007 6350 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.180.188" port: 43013 } has no permanent_uuid. Determining permanent_uuid...
I20260811 18:59:46.834693 6363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:46.834449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.188:43013 (local address 127.0.0.1:36790)
0811 18:59:46.834561 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:46.834612 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.188:43013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260811 18:59:46.835317 6363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:46.835068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.188:43013 (local address 127.0.0.1:36802)
0811 18:59:46.835209 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:46.835257 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.188:43013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
W20260811 18:59:46.835430 6350 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.180.188:43013: Network error: Client connection negotiation failed: client connection to 127.27.180.188:43013: connect: Connection refused (error 111)
I20260811 18:59:46.864540 6511 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.180.188" port: 43013 } attempt: 1
I20260811 18:59:46.865494 6515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:46.865125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.188:43013 (local address 127.0.0.1:36808)
0811 18:59:46.865320 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:46.865392 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.188:43013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260811 18:59:46.866567 6350 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.180.188" port: 43013 } attempt: 1
I20260811 18:59:46.867026 6515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:46.866667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.188:43013 (local address 127.0.0.1:36812)
0811 18:59:46.866876 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:46.866955 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.188:43013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117}
W20260811 18:59:46.867646 6511 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.180.188:43013: Network error: Client connection negotiation failed: client connection to 127.27.180.188:43013: connect: Connection refused (error 111)
I20260811 18:59:46.867967 6363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:46.866992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.188:43013 (local address 127.0.0.1:36820)
0811 18:59:46.867837 (+ 845us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:46.867892 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.188:43013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":758}
I20260811 18:59:46.868932 6363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:46.868710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.188:43013 (local address 127.0.0.1:36828)
0811 18:59:46.868813 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:46.868865 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.188:43013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260811 18:59:46.869146 6350 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.180.188:43013: Network error: Client connection negotiation failed: client connection to 127.27.180.188:43013: connect: Connection refused (error 111)
W20260811 18:59:46.929641 6523 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 18:59:46.929881 6523 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 18:59:46.929903 6523 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 18:59:46.929943 6523 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 18:59:46.933928 6523 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 18:59:46.933979 6523 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 18:59:46.933996 6523 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 18:59:46.934016 6523 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 18:59:46.934031 6523 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 18:59:46.935842 6511 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.180.188" port: 43013 } attempt: 2
I20260811 18:59:46.936646 6515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:46.936354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.188:43013 (local address 127.0.0.1:36838)
0811 18:59:46.936515 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:46.936572 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.188:43013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260811 18:59:46.937341 6515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:46.937138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.188:43013 (local address 127.0.0.1:36850)
0811 18:59:46.937238 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:46.937283 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.188:43013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260811 18:59:46.937454 6511 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.180.188:43013: Network error: Client connection negotiation failed: client connection to 127.27.180.188:43013: connect: Connection refused (error 111)
I20260811 18:59:46.939329 6523 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.27.180.148:44375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474496157106-28370-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474496157106-28370-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.27.180.190:40049,127.27.180.189:44959,127.27.180.188:43013
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474496157106-28370-0/krb5kdc/kudu_127.27.180.188.keytab
--principal=kudu/127.27.180.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.27.180.188:43013
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474496157106-28370-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.27.180.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6523
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474496157106-28370-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 18:11:32 UTC on ccycloud.quasar-bxyewg.root.comops.site
build id 4581128
I20260811 18:59:46.940284 6523 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 18:59:46.941262 6523 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 18:59:46.944298 6350 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.180.188" port: 43013 } attempt: 2
I20260811 18:59:46.944962 6363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:46.944713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.188:43013 (local address 127.0.0.1:36866)
0811 18:59:46.944838 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:46.944890 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.188:43013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260811 18:59:46.945757 6363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:46.945513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.188:43013 (local address 127.0.0.1:36870)
0811 18:59:46.945637 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:46.945692 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.27.180.188:43013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260811 18:59:46.945945 6350 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.27.180.188:43013: Network error: Client connection negotiation failed: client connection to 127.27.180.188:43013: connect: Connection refused (error 111)
W20260811 18:59:46.949697 6529 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:46.949707 6530 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 18:59:46.949734 6538 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 18:59:46.949743 6523 server_base.cc:1034] running on GCE node
I20260811 18:59:46.950397 6523 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 18:59:46.951435 6523 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 18:59:46.952605 6523 hybrid_clock.cc:648] HybridClock initialized: now 1786474786952581 us; error 34 us; skew 500 ppm
Aug 11 18:59:46 dist-test-slave-2j7r krb5kdc[6176](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474786, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 18:59:46.955226 6523 init.cc:377] Logged in from keytab as kudu/127.27.180.188@KRBTEST.COM (short username kudu)
I20260811 18:59:46.956037 6523 webserver.cc:460] Webserver started at http://127.27.180.188:37603/ using document root <none> and password file <none>
I20260811 18:59:46.956553 6523 fs_manager.cc:359] Metadata directory not provided
I20260811 18:59:46.956622 6523 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 18:59:46.956840 6523 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 18:59:46.958966 6523 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474496157106-28370-0/minicluster-data/master-2/data/instance:
uuid: "84bf0edc81a047eca5850155d3ad0228"
format_stamp: "Formatted at 2026-08-11 18:59:46 on dist-test-slave-2j7r"
server_key: "2f9bb3738b30e864a798d8b4feec177a"
server_key_iv: "c45ecb82f3dc2ba2ecab98c0d3e976b2"
server_key_version: "encryptionkey@0"
I20260811 18:59:46.959478 6523 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474496157106-28370-0/minicluster-data/master-2/wal/instance:
uuid: "84bf0edc81a047eca5850155d3ad0228"
format_stamp: "Formatted at 2026-08-11 18:59:46 on dist-test-slave-2j7r"
server_key: "2f9bb3738b30e864a798d8b4feec177a"
server_key_iv: "c45ecb82f3dc2ba2ecab98c0d3e976b2"
server_key_version: "encryptionkey@0"
I20260811 18:59:46.963034 6523 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 18:59:46.965325 6552 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 18:59:46.966502 6523 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 18:59:46.966609 6523 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474496157106-28370-0/minicluster-data/master-2/data,/tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474496157106-28370-0/minicluster-data/master-2/wal
uuid: "84bf0edc81a047eca5850155d3ad0228"
format_stamp: "Formatted at 2026-08-11 18:59:46 on dist-test-slave-2j7r"
server_key: "2f9bb3738b30e864a798d8b4feec177a"
server_key_iv: "c45ecb82f3dc2ba2ecab98c0d3e976b2"
server_key_version: "encryptionkey@0"
I20260811 18:59:46.966722 6523 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474496157106-28370-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474496157106-28370-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474496157106-28370-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 18:59:46.979843 6523 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 18:59:46.982583 6523 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 18:59:46.982748 6523 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 18:59:46.989491 6523 rpc_server.cc:307] RPC server started. Bound to: 127.27.180.188:43013
I20260811 18:59:46.989733 6664 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.27.180.188:43013 every 8 connection(s)
I20260811 18:59:46.990427 6523 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474496157106-28370-0/minicluster-data/master-2/data/info.pb
I20260811 18:59:46.991755 28370 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu as pid 6523
I20260811 18:59:46.991892 28370 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpMgJ1/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786474496157106-28370-0/minicluster-data/master-2/wal/instance
I20260811 18:59:46.992201 28370 external_mini_cluster.cc:1428] Setting key 05b19959a11ac24e8db2f29ed4c63d50
I20260811 18:59:46.993548 6668 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 18:59:46.997704 6668 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.180.190" port: 40049 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 18:59:46 dist-test-slave-2j7r krb5kdc[6176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474786, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.180.188@KRBTEST.COM
Aug 11 18:59:46 dist-test-slave-2j7r krb5kdc[6176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474786, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.188@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM
I20260811 18:59:47.006873 6671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:46.993508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36872 (local address 127.27.180.188:43013)
0811 18:59:46.993912 (+ 404us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:46.993928 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:46.993964 (+ 36us) server_negotiation.cc:407] Connection header received
0811 18:59:46.994598 (+ 634us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:46.994613 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:46.994944 (+ 331us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:46.995214 (+ 270us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:46.996124 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:46.997217 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:46.998331 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:46.998781 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:46.998905 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:47.001813 (+ 2908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:47.001820 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:47.001831 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:59:47.001862 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:47.003537 (+ 1675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:47.003957 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:47.003963 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:47.003968 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:47.004089 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:47.004791 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:47.004795 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:47.004796 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:47.005282 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:47.005438 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:47.005754 (+ 316us) server_negotiation.cc:299] Negotiation successful
0811 18:59:47.006047 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":263,"thread_start_us":114,"threads_started":1}
I20260811 18:59:47.008387 6673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:46.999439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.190:40049 (local address 127.0.0.1:35682)
0811 18:59:46.999853 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:46.999906 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 18:59:47.000032 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:59:47.000312 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:59:47.000318 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:59:47.000750 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:59:47.000881 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:47.000887 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:47.001856 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:47.001859 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:47.002773 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:47.002786 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:47.003206 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:47.003210 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:47.003322 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:47.003740 (+ 418us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:59:47.003752 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:59:47.005117 (+ 1365us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:59:47.006775 (+ 1658us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:59:47.006784 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:59:47.006789 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:59:47.007067 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:59:47.007351 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:59:47.007353 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:59:47.007354 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:59:47.007430 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:59:47.007825 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:59:47.007829 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:59:47.007931 (+ 102us) client_negotiation.cc:769] Sending connection context
0811 18:59:47.008042 (+ 111us) client_negotiation.cc:240] Negotiation successful
0811 18:59:47.008203 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":142,"threads_started":1}
I20260811 18:59:47.008392 6364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:46.999507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35682 (local address 127.27.180.190:40049)
0811 18:59:46.999646 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:46.999651 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:46.999989 (+ 338us) server_negotiation.cc:407] Connection header received
0811 18:59:47.000124 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:47.000127 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:47.000172 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:47.000237 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:47.000998 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:47.001740 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:47.002888 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:47.003098 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:47.003177 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:47.005256 (+ 2079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:47.005259 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:47.005262 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:59:47.005285 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:47.006673 (+ 1388us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:47.007165 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:47.007169 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:47.007170 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:47.007263 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:47.007509 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:47.007512 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:47.007514 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:47.007736 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:47.007835 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:47.008128 (+ 293us) server_negotiation.cc:299] Negotiation successful
0811 18:59:47.008236 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 18:59:47.009526 6668 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.180.189" port: 44959 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 18:59:47 dist-test-slave-2j7r krb5kdc[6176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474786, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.180.190@KRBTEST.COM
Aug 11 18:59:47 dist-test-slave-2j7r krb5kdc[6176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474786, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.188@KRBTEST.COM for kudu/127.27.180.189@KRBTEST.COM
I20260811 18:59:47.017374 6364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:47.009334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35686 (local address 127.27.180.190:40049)
0811 18:59:47.009443 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:47.009447 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:47.009461 (+ 14us) server_negotiation.cc:407] Connection header received
0811 18:59:47.009575 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:47.009577 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:47.009613 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:47.009682 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:47.010343 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:47.010975 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:47.011871 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:47.012189 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:47.012293 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:47.014464 (+ 2171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:47.014467 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:47.014470 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:59:47.014496 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:47.015840 (+ 1344us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:47.016269 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:47.016271 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:47.016272 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:47.016338 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:47.016597 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:47.016600 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:47.016601 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:47.016774 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:47.016994 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:47.017133 (+ 139us) server_negotiation.cc:299] Negotiation successful
0811 18:59:47.017231 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 18:59:47.017902 6673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:47.009971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.189:44959 (local address 127.0.0.1:33686)
0811 18:59:47.010105 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:47.010115 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 18:59:47.010168 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:59:47.010456 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:59:47.010459 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:59:47.010619 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:59:47.010719 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:47.010725 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:47.011556 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:47.011558 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:47.012194 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:47.012204 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:47.012737 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:47.012740 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:47.012820 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:47.013199 (+ 379us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:59:47.013210 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:59:47.014734 (+ 1524us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:59:47.016405 (+ 1671us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:59:47.016409 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:59:47.016411 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:59:47.016584 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:59:47.016824 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:59:47.016827 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:59:47.016829 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:59:47.016904 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:59:47.017453 (+ 549us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:59:47.017457 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:59:47.017545 (+ 88us) client_negotiation.cc:769] Sending connection context
0811 18:59:47.017640 (+ 95us) client_negotiation.cc:240] Negotiation successful
0811 18:59:47.017754 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":86}
I20260811 18:59:47.017946 6521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:47.010111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33686 (local address 127.27.180.189:44959)
0811 18:59:47.010241 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:47.010245 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:47.010259 (+ 14us) server_negotiation.cc:407] Connection header received
0811 18:59:47.010302 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:47.010305 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:47.010353 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:47.010427 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:47.010814 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:47.011447 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:47.012308 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:47.012617 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:47.012717 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:47.014857 (+ 2140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:47.014860 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:47.014862 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:59:47.014890 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:47.016312 (+ 1422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:47.016667 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:47.016670 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:47.016672 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:47.016744 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:47.017006 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:47.017009 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:47.017010 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:47.017256 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:47.017346 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:47.017697 (+ 351us) server_negotiation.cc:299] Negotiation successful
0811 18:59:47.017797 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 18:59:47.018637 6668 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.27.180.188" port: 43013 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 18:59:47 dist-test-slave-2j7r krb5kdc[6176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474786, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.27.180.189@KRBTEST.COM
Aug 11 18:59:47 dist-test-slave-2j7r krb5kdc[6176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474786, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.188@KRBTEST.COM for kudu/127.27.180.188@KRBTEST.COM
I20260811 18:59:47.026283 6521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:47.018496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33700 (local address 127.27.180.189:44959)
0811 18:59:47.018595 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:47.018599 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:47.018613 (+ 14us) server_negotiation.cc:407] Connection header received
0811 18:59:47.018714 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:47.018718 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:47.018757 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:47.018832 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:47.019454 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:47.020041 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:47.020742 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:47.021021 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:47.021118 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:47.023393 (+ 2275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:47.023396 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:47.023398 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:59:47.023418 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:47.024799 (+ 1381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:47.025205 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:47.025209 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:47.025210 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:47.025288 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:47.025534 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:47.025538 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:47.025539 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:47.025704 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:47.025947 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:47.026049 (+ 102us) server_negotiation.cc:299] Negotiation successful
0811 18:59:47.026145 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260811 18:59:47.026621 6673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:47.019095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.188:43013 (local address 127.0.0.1:36874)
0811 18:59:47.019186 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:47.019198 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 18:59:47.019273 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:59:47.019487 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:59:47.019491 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:59:47.019607 (+ 116us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:59:47.019675 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:47.019679 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:47.020901 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:47.020903 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:47.021576 (+ 673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:47.021588 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:47.022023 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:47.022028 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:47.022115 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:47.022447 (+ 332us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:59:47.022462 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:59:47.023767 (+ 1305us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:59:47.025278 (+ 1511us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:59:47.025281 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:59:47.025283 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:59:47.025454 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:59:47.025680 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:59:47.025683 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:59:47.025685 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:59:47.025754 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:59:47.026078 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:59:47.026081 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:59:47.026158 (+ 77us) client_negotiation.cc:769] Sending connection context
0811 18:59:47.026371 (+ 213us) client_negotiation.cc:240] Negotiation successful
0811 18:59:47.026490 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260811 18:59:47.027011 6671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:47.019125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36874 (local address 127.27.180.188:43013)
0811 18:59:47.019209 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:47.019212 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:47.019237 (+ 25us) server_negotiation.cc:407] Connection header received
0811 18:59:47.019334 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:47.019337 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:47.019383 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:47.019447 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:47.019766 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:47.020797 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:47.021693 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:47.021912 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:47.021986 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:47.023885 (+ 1899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:47.023887 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:47.023889 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:59:47.023908 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:47.025185 (+ 1277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:47.025532 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:47.025535 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:47.025537 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:47.025605 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:47.025827 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:47.025830 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:47.025832 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:47.025994 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:47.026073 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:47.026793 (+ 720us) server_negotiation.cc:299] Negotiation successful
0811 18:59:47.026917 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260811 18:59:47.028602 6511 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.180.188" port: 43013 } attempt: 3
I20260811 18:59:47.029819 6668 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84bf0edc81a047eca5850155d3ad0228: Bootstrap starting.
I20260811 18:59:47.031107 28370 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 18:59:47.032565 6668 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 84bf0edc81a047eca5850155d3ad0228: Neither blocks nor log segments found. Creating new log.
Aug 11 18:59:47 dist-test-slave-2j7r krb5kdc[6176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474786, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.189@KRBTEST.COM for kudu/127.27.180.188@KRBTEST.COM
I20260811 18:59:47.033457 6668 log.cc:824] T 00000000000000000000000000000000 P 84bf0edc81a047eca5850155d3ad0228: Log is configured to *not* fsync() on all Append() calls
I20260811 18:59:47.032269 6295 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "c150e5b86c96445c8d6e1bdf7dcb0d21"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35686
I20260811 18:59:47.035651 6668 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84bf0edc81a047eca5850155d3ad0228: No bootstrap required, opened a new log
I20260811 18:59:47.036146 6515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:47.028925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.188:43013 (local address 127.0.0.1:36886)
0811 18:59:47.029186 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:47.029197 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 18:59:47.029266 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:59:47.029456 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:59:47.029459 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:59:47.029620 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:59:47.029696 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:47.029702 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:47.030574 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:47.030578 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:47.031177 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:47.031186 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:47.031607 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:47.031610 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:47.031688 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:47.032046 (+ 358us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:59:47.032060 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:59:47.033080 (+ 1020us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:59:47.034700 (+ 1620us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:59:47.034704 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:59:47.034705 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:59:47.034948 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:59:47.035203 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:59:47.035206 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:59:47.035208 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:59:47.035301 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:59:47.035670 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:59:47.035673 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:59:47.035767 (+ 94us) client_negotiation.cc:769] Sending connection context
0811 18:59:47.035877 (+ 110us) client_negotiation.cc:240] Negotiation successful
0811 18:59:47.036005 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194}
I20260811 18:59:47.036194 6671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:47.029008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36886 (local address 127.27.180.188:43013)
0811 18:59:47.029128 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:47.029131 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:47.029248 (+ 117us) server_negotiation.cc:407] Connection header received
0811 18:59:47.029321 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:47.029323 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:47.029361 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:47.029435 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:47.029793 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:47.030450 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:47.031278 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:47.031512 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:47.031594 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:47.033261 (+ 1667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:47.033264 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:47.033266 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:59:47.033287 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:47.034597 (+ 1310us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:47.035036 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:47.035039 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:47.035041 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:47.035118 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:47.035394 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:47.035397 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:47.035399 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:47.035573 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:47.035655 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:47.035954 (+ 299us) server_negotiation.cc:299] Negotiation successful
0811 18:59:47.036056 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260811 18:59:47.038405 6511 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76d6f7110b1c42849a4198f8257dde74: Bootstrap starting.
I20260811 18:59:47.038839 6668 raft_consensus.cc:348] T 00000000000000000000000000000000 P 84bf0edc81a047eca5850155d3ad0228 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c150e5b86c96445c8d6e1bdf7dcb0d21" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 40049 } } peers { permanent_uuid: "76d6f7110b1c42849a4198f8257dde74" member_type: VOTER last_known_addr { host: "127.27.180.189" port: 44959 } } peers { permanent_uuid: "84bf0edc81a047eca5850155d3ad0228" member_type: VOTER last_known_addr { host: "127.27.180.188" port: 43013 } }
I20260811 18:59:47.039031 6668 raft_consensus.cc:729] T 00000000000000000000000000000000 P 84bf0edc81a047eca5850155d3ad0228 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84bf0edc81a047eca5850155d3ad0228, State: Initialized, Role: FOLLOWER
I20260811 18:59:47.039311 6668 consensus_queue.cc:260] T 00000000000000000000000000000000 P 84bf0edc81a047eca5850155d3ad0228 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c150e5b86c96445c8d6e1bdf7dcb0d21" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 40049 } } peers { permanent_uuid: "76d6f7110b1c42849a4198f8257dde74" member_type: VOTER last_known_addr { host: "127.27.180.189" port: 44959 } } peers { permanent_uuid: "84bf0edc81a047eca5850155d3ad0228" member_type: VOTER last_known_addr { host: "127.27.180.188" port: 43013 } }
I20260811 18:59:47.039741 6677 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84bf0edc81a047eca5850155d3ad0228 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c150e5b86c96445c8d6e1bdf7dcb0d21" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 40049 } } peers { permanent_uuid: "76d6f7110b1c42849a4198f8257dde74" member_type: VOTER last_known_addr { host: "127.27.180.189" port: 44959 } } peers { permanent_uuid: "84bf0edc81a047eca5850155d3ad0228" member_type: VOTER last_known_addr { host: "127.27.180.188" port: 43013 } } }
I20260811 18:59:47.039880 6677 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84bf0edc81a047eca5850155d3ad0228 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:59:47.040390 6668 sys_catalog.cc:564] T 00000000000000000000000000000000 P 84bf0edc81a047eca5850155d3ad0228 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:59:47.040571 6511 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 76d6f7110b1c42849a4198f8257dde74: Neither blocks nor log segments found. Creating new log.
I20260811 18:59:47.041292 6511 log.cc:824] T 00000000000000000000000000000000 P 76d6f7110b1c42849a4198f8257dde74: Log is configured to *not* fsync() on all Append() calls
I20260811 18:59:47.043161 6511 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76d6f7110b1c42849a4198f8257dde74: No bootstrap required, opened a new log
I20260811 18:59:47.045128 6350 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.27.180.188" port: 43013 } attempt: 3
I20260811 18:59:47.046451 6511 raft_consensus.cc:348] T 00000000000000000000000000000000 P 76d6f7110b1c42849a4198f8257dde74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c150e5b86c96445c8d6e1bdf7dcb0d21" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 40049 } } peers { permanent_uuid: "76d6f7110b1c42849a4198f8257dde74" member_type: VOTER last_known_addr { host: "127.27.180.189" port: 44959 } } peers { permanent_uuid: "84bf0edc81a047eca5850155d3ad0228" member_type: VOTER last_known_addr { host: "127.27.180.188" port: 43013 } }
I20260811 18:59:47.046728 6511 raft_consensus.cc:729] T 00000000000000000000000000000000 P 76d6f7110b1c42849a4198f8257dde74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76d6f7110b1c42849a4198f8257dde74, State: Initialized, Role: FOLLOWER
I20260811 18:59:47.047086 6511 consensus_queue.cc:260] T 00000000000000000000000000000000 P 76d6f7110b1c42849a4198f8257dde74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c150e5b86c96445c8d6e1bdf7dcb0d21" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 40049 } } peers { permanent_uuid: "76d6f7110b1c42849a4198f8257dde74" member_type: VOTER last_known_addr { host: "127.27.180.189" port: 44959 } } peers { permanent_uuid: "84bf0edc81a047eca5850155d3ad0228" member_type: VOTER last_known_addr { host: "127.27.180.188" port: 43013 } }
I20260811 18:59:47.048400 6511 sys_catalog.cc:564] T 00000000000000000000000000000000 P 76d6f7110b1c42849a4198f8257dde74 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 18:59:47.045110 6693 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 84bf0edc81a047eca5850155d3ad0228: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:59:47.049423 6693 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Aug 11 18:59:47 dist-test-slave-2j7r krb5kdc[6176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786474786, etypes {rep=17 tkt=17 ses=17}, kudu/127.27.180.190@KRBTEST.COM for kudu/127.27.180.188@KRBTEST.COM
I20260811 18:59:47.047178 6523 master_runner.cc:186] Error getting master registration for 127.27.180.190:40049: OK, instance_id { permanent_uuid: "c150e5b86c96445c8d6e1bdf7dcb0d21" instance_seqno: 1786474786584620 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
W20260811 18:59:47.054141 6709 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 76d6f7110b1c42849a4198f8257dde74: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260811 18:59:47.054409 6523 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 18:59:47.055564 6671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:47.045564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36888 (local address 127.27.180.188:43013)
0811 18:59:47.045658 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:47.045662 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:47.045791 (+ 129us) server_negotiation.cc:407] Connection header received
0811 18:59:47.045897 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:47.045900 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:47.045961 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:47.046041 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:47.046618 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:47.047316 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:47.048468 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:47.048795 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:47.048890 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:47.051405 (+ 2515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:47.051410 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:47.051412 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:59:47.051437 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:47.052983 (+ 1546us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:47.053547 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:47.053550 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:47.053553 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:47.053653 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:47.054090 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:47.054094 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:47.054096 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:47.054638 (+ 542us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:47.055017 (+ 379us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:47.055287 (+ 270us) server_negotiation.cc:299] Negotiation successful
0811 18:59:47.055414 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260811 18:59:47.055673 6363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:47.045519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.27.180.188:43013 (local address 127.0.0.1:36888)
0811 18:59:47.045718 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0811 18:59:47.045731 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 18:59:47.045817 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 18:59:47.046176 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 18:59:47.046180 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 18:59:47.046360 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0811 18:59:47.046472 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:47.046479 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:47.047505 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:47.047509 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:47.048302 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 18:59:47.048314 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 18:59:47.049031 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 18:59:47.049036 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 18:59:47.049142 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:47.049643 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 18:59:47.049660 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 18:59:47.051208 (+ 1548us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 18:59:47.053150 (+ 1942us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:59:47.053153 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:59:47.053155 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:59:47.053418 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:59:47.053769 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 18:59:47.053773 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 18:59:47.053774 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 18:59:47.053940 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 18:59:47.054815 (+ 875us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 18:59:47.054818 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 18:59:47.054964 (+ 146us) client_negotiation.cc:769] Sending connection context
0811 18:59:47.055216 (+ 252us) client_negotiation.cc:240] Negotiation successful
0811 18:59:47.055426 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130}
I20260811 18:59:47.051869 6373 master_runner.cc:186] Error getting master registration for 127.27.180.190:40049: OK, instance_id { permanent_uuid: "c150e5b86c96445c8d6e1bdf7dcb0d21" instance_seqno: 1786474786584620 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
W20260811 18:59:47.057201 6709 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:59:47.057397 6373 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 18:59:47.059129 6350 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c150e5b86c96445c8d6e1bdf7dcb0d21: Bootstrap starting.
I20260811 18:59:47.061363 6696 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76d6f7110b1c42849a4198f8257dde74 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c150e5b86c96445c8d6e1bdf7dcb0d21" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 40049 } } peers { permanent_uuid: "76d6f7110b1c42849a4198f8257dde74" member_type: VOTER last_known_addr { host: "127.27.180.189" port: 44959 } } peers { permanent_uuid: "84bf0edc81a047eca5850155d3ad0228" member_type: VOTER last_known_addr { host: "127.27.180.188" port: 43013 } } }
I20260811 18:59:47.061517 6696 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76d6f7110b1c42849a4198f8257dde74 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:59:47.063326 6350 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c150e5b86c96445c8d6e1bdf7dcb0d21: Neither blocks nor log segments found. Creating new log.
I20260811 18:59:47.064371 6350 log.cc:824] T 00000000000000000000000000000000 P c150e5b86c96445c8d6e1bdf7dcb0d21: Log is configured to *not* fsync() on all Append() calls
I20260811 18:59:47.067412 6350 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c150e5b86c96445c8d6e1bdf7dcb0d21: No bootstrap required, opened a new log
I20260811 18:59:47.072053 6350 raft_consensus.cc:348] T 00000000000000000000000000000000 P c150e5b86c96445c8d6e1bdf7dcb0d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c150e5b86c96445c8d6e1bdf7dcb0d21" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 40049 } } peers { permanent_uuid: "76d6f7110b1c42849a4198f8257dde74" member_type: VOTER last_known_addr { host: "127.27.180.189" port: 44959 } } peers { permanent_uuid: "84bf0edc81a047eca5850155d3ad0228" member_type: VOTER last_known_addr { host: "127.27.180.188" port: 43013 } }
I20260811 18:59:47.072409 6350 raft_consensus.cc:729] T 00000000000000000000000000000000 P c150e5b86c96445c8d6e1bdf7dcb0d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c150e5b86c96445c8d6e1bdf7dcb0d21, State: Initialized, Role: FOLLOWER
I20260811 18:59:47.072913 6350 consensus_queue.cc:260] T 00000000000000000000000000000000 P c150e5b86c96445c8d6e1bdf7dcb0d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c150e5b86c96445c8d6e1bdf7dcb0d21" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 40049 } } peers { permanent_uuid: "76d6f7110b1c42849a4198f8257dde74" member_type: VOTER last_known_addr { host: "127.27.180.189" port: 44959 } } peers { permanent_uuid: "84bf0edc81a047eca5850155d3ad0228" member_type: VOTER last_known_addr { host: "127.27.180.188" port: 43013 } }
I20260811 18:59:47.073632 6714 sys_catalog.cc:455] T 00000000000000000000000000000000 P c150e5b86c96445c8d6e1bdf7dcb0d21 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c150e5b86c96445c8d6e1bdf7dcb0d21" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 40049 } } peers { permanent_uuid: "76d6f7110b1c42849a4198f8257dde74" member_type: VOTER last_known_addr { host: "127.27.180.189" port: 44959 } } peers { permanent_uuid: "84bf0edc81a047eca5850155d3ad0228" member_type: VOTER last_known_addr { host: "127.27.180.188" port: 43013 } } }
I20260811 18:59:47.073771 6714 sys_catalog.cc:458] T 00000000000000000000000000000000 P c150e5b86c96445c8d6e1bdf7dcb0d21 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:59:47.075103 6295 raft_consensus.cc:482] T 00000000000000000000000000000000 P c150e5b86c96445c8d6e1bdf7dcb0d21 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260811 18:59:47.075214 6295 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c150e5b86c96445c8d6e1bdf7dcb0d21 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:59:47.076426 6295 raft_consensus.cc:504] T 00000000000000000000000000000000 P c150e5b86c96445c8d6e1bdf7dcb0d21 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c150e5b86c96445c8d6e1bdf7dcb0d21" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 40049 } } peers { permanent_uuid: "76d6f7110b1c42849a4198f8257dde74" member_type: VOTER last_known_addr { host: "127.27.180.189" port: 44959 } } peers { permanent_uuid: "84bf0edc81a047eca5850155d3ad0228" member_type: VOTER last_known_addr { host: "127.27.180.188" port: 43013 } }
I20260811 18:59:47.077251 6350 sys_catalog.cc:564] T 00000000000000000000000000000000 P c150e5b86c96445c8d6e1bdf7dcb0d21 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 18:59:47.077386 6295 leader_election.cc:288] T 00000000000000000000000000000000 P c150e5b86c96445c8d6e1bdf7dcb0d21 [CANDIDATE]: Term 1 election: Requested vote from peers 76d6f7110b1c42849a4198f8257dde74 (127.27.180.189:44959), 84bf0edc81a047eca5850155d3ad0228 (127.27.180.188:43013)
I20260811 18:59:47.078657 6463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c150e5b86c96445c8d6e1bdf7dcb0d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "76d6f7110b1c42849a4198f8257dde74"
I20260811 18:59:47.078927 6463 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 76d6f7110b1c42849a4198f8257dde74 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:59:47.080240 6463 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 76d6f7110b1c42849a4198f8257dde74 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c150e5b86c96445c8d6e1bdf7dcb0d21 in term 1.
I20260811 18:59:47.081557 6615 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c150e5b86c96445c8d6e1bdf7dcb0d21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "84bf0edc81a047eca5850155d3ad0228"
I20260811 18:59:47.081784 6615 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 84bf0edc81a047eca5850155d3ad0228 [term 0 FOLLOWER]: Advancing to term 1
I20260811 18:59:47.083132 6615 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 84bf0edc81a047eca5850155d3ad0228 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c150e5b86c96445c8d6e1bdf7dcb0d21 in term 1.
I20260811 18:59:47.080693 6236 leader_election.cc:302] T 00000000000000000000000000000000 P c150e5b86c96445c8d6e1bdf7dcb0d21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76d6f7110b1c42849a4198f8257dde74, c150e5b86c96445c8d6e1bdf7dcb0d21; no voters:
I20260811 18:59:47.084214 6714 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c150e5b86c96445c8d6e1bdf7dcb0d21 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 18:59:47.084451 6714 raft_consensus.cc:686] T 00000000000000000000000000000000 P c150e5b86c96445c8d6e1bdf7dcb0d21 [term 1 LEADER]: Becoming Leader. State: Replica: c150e5b86c96445c8d6e1bdf7dcb0d21, State: Running, Role: LEADER
I20260811 18:59:47.084784 6714 consensus_queue.cc:237] T 00000000000000000000000000000000 P c150e5b86c96445c8d6e1bdf7dcb0d21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c150e5b86c96445c8d6e1bdf7dcb0d21" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 40049 } } peers { permanent_uuid: "76d6f7110b1c42849a4198f8257dde74" member_type: VOTER last_known_addr { host: "127.27.180.189" port: 44959 } } peers { permanent_uuid: "84bf0edc81a047eca5850155d3ad0228" member_type: VOTER last_known_addr { host: "127.27.180.188" port: 43013 } }
W20260811 18:59:47.081172 6246 tablet.cc:2369] T 00000000000000000000000000000000 P c150e5b86c96445c8d6e1bdf7dcb0d21: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 18:59:47.089815 6735 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c150e5b86c96445c8d6e1bdf7dcb0d21: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 18:59:47.089895 6735 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 18:59:47.092726 6750 sys_catalog.cc:455] T 00000000000000000000000000000000 P c150e5b86c96445c8d6e1bdf7dcb0d21 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c150e5b86c96445c8d6e1bdf7dcb0d21. Latest consensus state: current_term: 1 leader_uuid: "c150e5b86c96445c8d6e1bdf7dcb0d21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c150e5b86c96445c8d6e1bdf7dcb0d21" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 40049 } } peers { permanent_uuid: "76d6f7110b1c42849a4198f8257dde74" member_type: VOTER last_known_addr { host: "127.27.180.189" port: 44959 } } peers { permanent_uuid: "84bf0edc81a047eca5850155d3ad0228" member_type: VOTER last_known_addr { host: "127.27.180.188" port: 43013 } } }
I20260811 18:59:47.092903 6750 sys_catalog.cc:458] T 00000000000000000000000000000000 P c150e5b86c96445c8d6e1bdf7dcb0d21 [sys.catalog]: This master's current role is: LEADER
I20260811 18:59:47.098047 6756 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 18:59:47.099771 6671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:47.082976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36890 (local address 127.27.180.188:43013)
0811 18:59:47.083082 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:47.083086 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:47.090186 (+ 7100us) server_negotiation.cc:407] Connection header received
0811 18:59:47.090356 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:47.090359 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:47.090422 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:47.090471 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:47.091289 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:47.092075 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:47.093269 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:47.093617 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:47.093686 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:47.095466 (+ 1780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:47.095470 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:47.095472 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:59:47.095499 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:47.097446 (+ 1947us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:47.098143 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:47.098147 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:47.098149 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:47.098239 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:47.098794 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:47.098798 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:47.098800 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:47.099020 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:47.099141 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:47.099495 (+ 354us) server_negotiation.cc:299] Negotiation successful
0811 18:59:47.099611 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 18:59:47.101006 6756 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 18:59:47.102181 6521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:47.082856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33712 (local address 127.27.180.189:44959)
0811 18:59:47.082960 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:47.082964 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:47.085736 (+ 2772us) server_negotiation.cc:407] Connection header received
0811 18:59:47.085840 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:47.085843 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:47.085901 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:47.085995 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:47.093890 (+ 7895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:47.094586 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:47.095601 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:47.095916 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:47.095984 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:47.098767 (+ 2783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:47.098771 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:47.098774 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:59:47.098801 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:47.100356 (+ 1555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:47.100824 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:47.100827 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:47.100829 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:47.100933 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:47.101255 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:47.101258 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:47.101260 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:47.101474 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:47.101576 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:47.101917 (+ 341us) server_negotiation.cc:299] Negotiation successful
0811 18:59:47.102037 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260811 18:59:47.102286 6364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:47.081839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35700 (local address 127.27.180.190:40049)
0811 18:59:47.081949 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:47.081954 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:47.085390 (+ 3436us) server_negotiation.cc:407] Connection header received
0811 18:59:47.085971 (+ 581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:47.085974 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:47.086021 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:47.086068 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:47.093662 (+ 7594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:47.094395 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:47.095487 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:47.095787 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:47.095852 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:47.097661 (+ 1809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 18:59:47.097665 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 18:59:47.097668 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 18:59:47.097693 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 18:59:47.099226 (+ 1533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:47.099714 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:47.099718 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:47.099720 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:47.099814 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 18:59:47.101427 (+ 1613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 18:59:47.101430 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 18:59:47.101432 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 18:59:47.101632 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 18:59:47.101741 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:47.102052 (+ 311us) server_negotiation.cc:299] Negotiation successful
0811 18:59:47.102155 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 18:59:47.110677 6615 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 84bf0edc81a047eca5850155d3ad0228 [term 1 FOLLOWER]: Refusing update from remote peer c150e5b86c96445c8d6e1bdf7dcb0d21: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:59:47.111454 6750 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c150e5b86c96445c8d6e1bdf7dcb0d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84bf0edc81a047eca5850155d3ad0228" member_type: VOTER last_known_addr { host: "127.27.180.188" port: 43013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:59:47.115818 6677 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84bf0edc81a047eca5850155d3ad0228 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c150e5b86c96445c8d6e1bdf7dcb0d21. Latest consensus state: current_term: 1 leader_uuid: "c150e5b86c96445c8d6e1bdf7dcb0d21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c150e5b86c96445c8d6e1bdf7dcb0d21" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 40049 } } peers { permanent_uuid: "76d6f7110b1c42849a4198f8257dde74" member_type: VOTER last_known_addr { host: "127.27.180.189" port: 44959 } } peers { permanent_uuid: "84bf0edc81a047eca5850155d3ad0228" member_type: VOTER last_known_addr { host: "127.27.180.188" port: 43013 } } }
I20260811 18:59:47.115931 6677 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84bf0edc81a047eca5850155d3ad0228 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:59:47.118672 6463 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 76d6f7110b1c42849a4198f8257dde74 [term 1 FOLLOWER]: Refusing update from remote peer c150e5b86c96445c8d6e1bdf7dcb0d21: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 18:59:47.119244 6750 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c150e5b86c96445c8d6e1bdf7dcb0d21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "76d6f7110b1c42849a4198f8257dde74" member_type: VOTER last_known_addr { host: "127.27.180.189" port: 44959 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 18:59:47.123113 6696 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76d6f7110b1c42849a4198f8257dde74 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c150e5b86c96445c8d6e1bdf7dcb0d21. Latest consensus state: current_term: 1 leader_uuid: "c150e5b86c96445c8d6e1bdf7dcb0d21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c150e5b86c96445c8d6e1bdf7dcb0d21" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 40049 } } peers { permanent_uuid: "76d6f7110b1c42849a4198f8257dde74" member_type: VOTER last_known_addr { host: "127.27.180.189" port: 44959 } } peers { permanent_uuid: "84bf0edc81a047eca5850155d3ad0228" member_type: VOTER last_known_addr { host: "127.27.180.188" port: 43013 } } }
I20260811 18:59:47.123238 6696 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76d6f7110b1c42849a4198f8257dde74 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:59:47.133033 6750 sys_catalog.cc:455] T 00000000000000000000000000000000 P c150e5b86c96445c8d6e1bdf7dcb0d21 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c150e5b86c96445c8d6e1bdf7dcb0d21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c150e5b86c96445c8d6e1bdf7dcb0d21" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 40049 } } peers { permanent_uuid: "76d6f7110b1c42849a4198f8257dde74" member_type: VOTER last_known_addr { host: "127.27.180.189" port: 44959 } } peers { permanent_uuid: "84bf0edc81a047eca5850155d3ad0228" member_type: VOTER last_known_addr { host: "127.27.180.188" port: 43013 } } }
I20260811 18:59:47.133208 6750 sys_catalog.cc:458] T 00000000000000000000000000000000 P c150e5b86c96445c8d6e1bdf7dcb0d21 [sys.catalog]: This master's current role is: LEADER
I20260811 18:59:47.136011 6696 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76d6f7110b1c42849a4198f8257dde74 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c150e5b86c96445c8d6e1bdf7dcb0d21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c150e5b86c96445c8d6e1bdf7dcb0d21" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 40049 } } peers { permanent_uuid: "76d6f7110b1c42849a4198f8257dde74" member_type: VOTER last_known_addr { host: "127.27.180.189" port: 44959 } } peers { permanent_uuid: "84bf0edc81a047eca5850155d3ad0228" member_type: VOTER last_known_addr { host: "127.27.180.188" port: 43013 } } }
I20260811 18:59:47.136121 6696 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76d6f7110b1c42849a4198f8257dde74 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:59:47.136904 6757 mvcc.cc:205] Tried to move back new op lower bound from 7317400727998693376 to 7317400727902818304. Current Snapshot: MvccSnapshot[applied={T|T < 7317400727998693376}]
I20260811 18:59:47.138250 6769 mvcc.cc:205] Tried to move back new op lower bound from 7317400727998693376 to 7317400727902818304. Current Snapshot: MvccSnapshot[applied={T|T < 7317400727998693376}]
I20260811 18:59:47.138985 6714 sys_catalog.cc:455] T 00000000000000000000000000000000 P c150e5b86c96445c8d6e1bdf7dcb0d21 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c150e5b86c96445c8d6e1bdf7dcb0d21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c150e5b86c96445c8d6e1bdf7dcb0d21" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 40049 } } peers { permanent_uuid: "76d6f7110b1c42849a4198f8257dde74" member_type: VOTER last_known_addr { host: "127.27.180.189" port: 44959 } } peers { permanent_uuid: "84bf0edc81a047eca5850155d3ad0228" member_type: VOTER last_known_addr { host: "127.27.180.188" port: 43013 } } }
I20260811 18:59:47.139099 6714 sys_catalog.cc:458] T 00000000000000000000000000000000 P c150e5b86c96445c8d6e1bdf7dcb0d21 [sys.catalog]: This master's current role is: LEADER
I20260811 18:59:47.140239 6677 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84bf0edc81a047eca5850155d3ad0228 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c150e5b86c96445c8d6e1bdf7dcb0d21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c150e5b86c96445c8d6e1bdf7dcb0d21" member_type: VOTER last_known_addr { host: "127.27.180.190" port: 40049 } } peers { permanent_uuid: "76d6f7110b1c42849a4198f8257dde74" member_type: VOTER last_known_addr { host: "127.27.180.189" port: 44959 } } peers { permanent_uuid: "84bf0edc81a047eca5850155d3ad0228" member_type: VOTER last_known_addr { host: "127.27.180.188" port: 43013 } } }
I20260811 18:59:47.140349 6677 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84bf0edc81a047eca5850155d3ad0228 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 18:59:47.141171 6759 mvcc.cc:205] Tried to move back new op lower bound from 7317400727998693376 to 7317400727902818304. Current Snapshot: MvccSnapshot[applied={T|T < 7317400727998693376}]
I20260811 18:59:47.141955 6756 catalog_manager.cc:1347] Generated new cluster ID: e0f1f8cc5f4146d2a6a58f8a84d1b6c3
I20260811 18:59:47.142025 6756 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 18:59:47.158931 6756 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 18:59:47.162645 6756 catalog_manager.cc:1504] Loading token signing keys...
I20260811 18:59:47.178238 6756 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c150e5b86c96445c8d6e1bdf7dcb0d21: Generated new TSK 0
I20260811 18:59:47.179128 6756 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 18:59:47.216878 6364 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:59:47.213607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35714 (local address 127.27.180.190:40049)
0811 18:59:47.213743 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:47.213748 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:47.215746 (+ 1998us) server_negotiation.cc:407] Connection header received
0811 18:59:47.215945 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:47.215949 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:47.216002 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:47.216054 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:47.216728 (+ 674us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35714: BlockingRecv error: recv got EOF from 127.0.0.1:35714 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260811 18:59:47.218927 6521 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:59:47.214481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33724 (local address 127.27.180.189:44959)
0811 18:59:47.214598 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:47.214602 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:47.217611 (+ 3009us) server_negotiation.cc:407] Connection header received
0811 18:59:47.218110 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:47.218113 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:47.218167 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:47.218218 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:47.218788 (+ 570us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33724: BlockingRecv error: recv got EOF from 127.0.0.1:33724 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260811 18:59:47.220165 6671 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:59:47.215130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36898 (local address 127.27.180.188:43013)
0811 18:59:47.215243 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:47.215247 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:47.217836 (+ 2589us) server_negotiation.cc:407] Connection header received
0811 18:59:47.219352 (+ 1516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:47.219369 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:47.219431 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:47.219484 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:47.220041 (+ 557us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36898: BlockingRecv error: recv got EOF from 127.0.0.1:36898 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260811 18:59:47.225386 6521 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:59:47.223695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33734 (local address 127.27.180.189:44959)
0811 18:59:47.223804 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:47.223809 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:47.224292 (+ 483us) server_negotiation.cc:407] Connection header received
0811 18:59:47.224550 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:47.224554 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:47.224636 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:47.224691 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:47.225261 (+ 570us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33734: BlockingRecv error: recv got EOF from 127.0.0.1:33734 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260811 18:59:47.230948 6521 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:59:47.229370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33748 (local address 127.27.180.189:44959)
0811 18:59:47.229484 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:47.229488 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:47.230035 (+ 547us) server_negotiation.cc:407] Connection header received
0811 18:59:47.230218 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:47.230221 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:47.230276 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:47.230326 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:47.230837 (+ 511us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33748: BlockingRecv error: recv got EOF from 127.0.0.1:33748 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260811 18:59:47.236864 6521 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:59:47.235317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33752 (local address 127.27.180.189:44959)
0811 18:59:47.235441 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:47.235445 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:47.235944 (+ 499us) server_negotiation.cc:407] Connection header received
0811 18:59:47.236131 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:47.236135 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:47.236187 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:47.236237 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:47.236757 (+ 520us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33752: BlockingRecv error: recv got EOF from 127.0.0.1:33752 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260811 18:59:47.244364 6521 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:59:47.243237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33756 (local address 127.27.180.189:44959)
0811 18:59:47.243368 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:47.243372 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:47.243547 (+ 175us) server_negotiation.cc:407] Connection header received
0811 18:59:47.243675 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:47.243678 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:47.243732 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:47.243826 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:47.244259 (+ 433us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33756: BlockingRecv error: recv got EOF from 127.0.0.1:33756 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
W20260811 18:59:47.255930 6521 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:59:47.254650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33770 (local address 127.27.180.189:44959)
0811 18:59:47.254763 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:47.254767 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:47.255061 (+ 294us) server_negotiation.cc:407] Connection header received
0811 18:59:47.255185 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:47.255188 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:47.255252 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:47.255340 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:47.255809 (+ 469us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33770: BlockingRecv error: recv got EOF from 127.0.0.1:33770 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260811 18:59:47.275065 6521 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:59:47.273883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33786 (local address 127.27.180.189:44959)
0811 18:59:47.274008 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:47.274013 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:47.274191 (+ 178us) server_negotiation.cc:407] Connection header received
0811 18:59:47.274339 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:47.274342 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:47.274392 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:47.274462 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:47.274940 (+ 478us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33786: BlockingRecv error: recv got EOF from 127.0.0.1:33786 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260811 18:59:47.310405 6521 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:59:47.309226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33802 (local address 127.27.180.189:44959)
0811 18:59:47.309388 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:47.309393 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:47.309526 (+ 133us) server_negotiation.cc:407] Connection header received
0811 18:59:47.309686 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:47.309688 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:47.309738 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:47.309811 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:47.310299 (+ 488us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33802: BlockingRecv error: recv got EOF from 127.0.0.1:33802 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260811 18:59:47.377877 6521 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:59:47.376650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33818 (local address 127.27.180.189:44959)
0811 18:59:47.376810 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:47.376814 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:47.376944 (+ 130us) server_negotiation.cc:407] Connection header received
0811 18:59:47.377131 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:47.377135 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:47.377192 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:47.377267 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:47.377761 (+ 494us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33818: BlockingRecv error: recv got EOF from 127.0.0.1:33818 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260811 18:59:47.509196 6521 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:59:47.508129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33830 (local address 127.27.180.189:44959)
0811 18:59:47.508312 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:47.508316 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:47.508406 (+ 90us) server_negotiation.cc:407] Connection header received
0811 18:59:47.508525 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:47.508528 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:47.508577 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:47.508648 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:47.509038 (+ 390us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33830: BlockingRecv error: recv got EOF from 127.0.0.1:33830 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
W20260811 18:59:47.768283 6521 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 18:59:47.767217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33844 (local address 127.27.180.189:44959)
0811 18:59:47.767401 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:47.767405 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:47.767425 (+ 20us) server_negotiation.cc:407] Connection header received
0811 18:59:47.767558 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:47.767561 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:47.767610 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:47.767681 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 18:59:47.768117 (+ 436us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33844: BlockingRecv error: recv got EOF from 127.0.0.1:33844 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
I20260811 18:59:48.057261 6693 catalog_manager.cc:1259] Loaded cluster ID: e0f1f8cc5f4146d2a6a58f8a84d1b6c3
I20260811 18:59:48.057337 6693 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 84bf0edc81a047eca5850155d3ad0228: loading cluster ID for follower catalog manager: success
I20260811 18:59:48.058429 6709 catalog_manager.cc:1259] Loaded cluster ID: e0f1f8cc5f4146d2a6a58f8a84d1b6c3
I20260811 18:59:48.058490 6709 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 76d6f7110b1c42849a4198f8257dde74: loading cluster ID for follower catalog manager: success
I20260811 18:59:48.059064 6693 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 84bf0edc81a047eca5850155d3ad0228: acquiring CA information for follower catalog manager: success
I20260811 18:59:48.059978 6709 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 76d6f7110b1c42849a4198f8257dde74: acquiring CA information for follower catalog manager: success
I20260811 18:59:48.060196 6693 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 84bf0edc81a047eca5850155d3ad0228: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 18:59:48.060981 6709 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 76d6f7110b1c42849a4198f8257dde74: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 18:59:48.285640 6901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:48.282199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33856 (local address 127.27.180.189:44959)
0811 18:59:48.282431 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:48.282435 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:48.282490 (+ 55us) server_negotiation.cc:407] Connection header received
0811 18:59:48.282612 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:48.282615 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:48.282746 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:48.282849 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:59:48.283052 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:48.283571 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:48.284335 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:48.284547 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:48.284613 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:48.284844 (+ 231us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:59:48.285325 (+ 481us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:59:48.285420 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:48.285464 (+ 44us) server_negotiation.cc:299] Negotiation successful
0811 18:59:48.285514 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":67,"threads_started":1}
I20260811 18:59:48.291895 6909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:48.288471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36902 (local address 127.27.180.188:43013)
0811 18:59:48.288711 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:48.288715 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:48.288728 (+ 13us) server_negotiation.cc:407] Connection header received
0811 18:59:48.288806 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:48.288808 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:48.288926 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:48.289021 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:59:48.289258 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:48.289760 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:48.290547 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:48.290771 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:48.290856 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:48.291134 (+ 278us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:59:48.291506 (+ 372us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:59:48.291611 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:48.291691 (+ 80us) server_negotiation.cc:299] Negotiation successful
0811 18:59:48.291746 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":65,"threads_started":1}
I20260811 18:59:48.299430 6917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:48.294819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35716 (local address 127.27.180.190:40049)
0811 18:59:48.295063 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:48.295069 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:48.295229 (+ 160us) server_negotiation.cc:407] Connection header received
0811 18:59:48.295431 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:48.295434 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:48.295592 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:48.295710 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:59:48.296022 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:48.296870 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:48.298014 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:48.298242 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:48.298329 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:48.298640 (+ 311us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:59:48.298979 (+ 339us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:59:48.299066 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:48.299214 (+ 148us) server_negotiation.cc:299] Negotiation successful
0811 18:59:48.299272 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":67,"threads_started":1}
I20260811 18:59:48.299538 6901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:48.294749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33872 (local address 127.27.180.189:44959)
0811 18:59:48.294916 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:48.294921 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:48.295174 (+ 253us) server_negotiation.cc:407] Connection header received
0811 18:59:48.295389 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:48.295392 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:48.295465 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:48.295534 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:59:48.295934 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:48.296935 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:48.298408 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:48.298684 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:48.298769 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:48.299056 (+ 287us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:59:48.299136 (+ 80us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:59:48.299181 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:48.299347 (+ 166us) server_negotiation.cc:299] Negotiation successful
0811 18:59:48.299412 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260811 18:59:48.299609 6909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 18:59:48.295425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36906 (local address 127.27.180.188:43013)
0811 18:59:48.295562 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 18:59:48.295584 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 18:59:48.296053 (+ 469us) server_negotiation.cc:407] Connection header received
0811 18:59:48.296278 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 18:59:48.296282 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 18:59:48.296347 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 18:59:48.296441 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 18:59:48.296666 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:48.297507 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:48.298462 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 18:59:48.298749 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 18:59:48.298840 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 18:59:48.299193 (+ 353us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 18:59:48.299267 (+ 74us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 18:59:48.299335 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 18:59:48.299456 (+ 121us) server_negotiation.cc:299] Negotiation successful
0811 18:59:48.299504 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 18:59:48.302845 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 6200
I20260811 18:59:48.315184 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 6373
I20260811 18:59:48.329906 28370 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpMgJ1/build/debug/bin/kudu with pid 6523
2026-08-11T18:59:48Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3236 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6485 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (292138 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST